2026-02-22T20:28:56.320 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-22T20:28:56.336 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-22T20:28:56.452 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/nmordech-2026-02-22_19:44:01-rados-wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle-distro-default-trial/64522 branch: wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle description: rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async objectstore/{bluestore/{alloc$/{hybrid} base mem$/{low} onode-segment$/{1M} write$/{v2/{compr$/{no$/{no}} v2}}}} rados supported-random-distro$/{rpm_latest}} email: null first_in_suite: false flavor: default job_id: '64522' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: nmordech-2026-02-22_19:44:01-rados-wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle-distro-default-trial no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: rocky os_version: '10.1' overrides: admin_socket: branch: wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle ceph: conf: global: mon election default strategy: 1 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore onode segment size: 1024K bluestore write v2: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd mclock skip benchmark: true osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: a5c3084f688b1f6eff223b8d259a8316f9844c8e ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: distro-suffix: rockylinux-10 install: ceph: flavor: default sha1: a5c3084f688b1f6eff223b8d259a8316f9844c8e selinux: allowlist: - comm="systemd".*denied.*\{ prog_run \}.*tclass=bpf.*permissive=1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle sha1: a5c3084f688b1f6eff223b8d259a8316f9844c8e owner: scheduled_nmordech@soko04.front.sepia.ceph.com priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 - osd.3 seed: 966 sha1: a5c3084f688b1f6eff223b8d259a8316f9844c8e sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a5c3084f688b1f6eff223b8d259a8316f9844c8e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a5c3084f688b1f6eff223b8d259a8316f9844c8e targets: trial084.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB7B7+qKjlR5tZuvVbl/t06SF9cyUM9elTViDNxRqMTJlt+ehCvw88ZNycOYou0wy6tA7AhFaj7ixMy0W0pk8J4= tasks: - install: null - ceph: conf: osd: osd min pg log entries: 5 osd_fast_fail_on_connection_refused: false log-ignorelist: - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - osd_recovery.test_incomplete_pgs: null - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-22_19:44:01 tube: trial user: nmordech verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653 2026-02-22T20:28:56.452 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a5c3084f688b1f6eff223b8d259a8316f9844c8e/qa; will attempt to use it 2026-02-22T20:28:56.452 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a5c3084f688b1f6eff223b8d259a8316f9844c8e/qa/tasks 2026-02-22T20:28:56.452 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-22T20:28:56.456 INFO:teuthology.task.internal:Checking packages... 2026-02-22T20:28:56.456 INFO:teuthology.task.internal:Checking packages for os_type 'rocky', flavor 'default' and ceph hash 'a5c3084f688b1f6eff223b8d259a8316f9844c8e' 2026-02-22T20:28:56.456 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-22T20:28:56.456 INFO:teuthology.packaging:ref: None 2026-02-22T20:28:56.456 INFO:teuthology.packaging:tag: None 2026-02-22T20:28:56.456 INFO:teuthology.packaging:branch: wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle 2026-02-22T20:28:56.456 INFO:teuthology.packaging:sha1: a5c3084f688b1f6eff223b8d259a8316f9844c8e 2026-02-22T20:28:56.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=rocky%2F10%2Fx86_64&ref=wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle 2026-02-22T20:28:56.575 INFO:teuthology.task.internal:Found packages for ceph version 20.2.0-794.ga5c3084f 2026-02-22T20:28:56.577 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-22T20:28:56.588 INFO:teuthology.task.internal:no buildpackages task found 2026-02-22T20:28:56.588 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-22T20:28:56.601 INFO:teuthology.task.internal:Saving configuration 2026-02-22T20:28:56.612 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-22T20:28:56.627 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-22T20:28:56.704 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial084.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/nmordech-2026-02-22_19:44:01-rados-wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle-distro-default-trial/64522', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'rocky', 'os_version': '10.1', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-22 20:25:28.056241', 'locked_by': 'scheduled_nmordech@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB7B7+qKjlR5tZuvVbl/t06SF9cyUM9elTViDNxRqMTJlt+ehCvw88ZNycOYou0wy6tA7AhFaj7ixMy0W0pk8J4='} 2026-02-22T20:28:56.704 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-22T20:28:56.719 INFO:teuthology.task.internal:roles: ubuntu@trial084.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2026-02-22T20:28:56.719 INFO:teuthology.run_tasks:Running task console_log... 2026-02-22T20:28:56.799 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x778de73fa7a0>, signals=[15]) 2026-02-22T20:28:56.799 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-22T20:28:56.809 INFO:teuthology.task.internal:Opening connections... 2026-02-22T20:28:56.809 DEBUG:teuthology.task.internal:connecting to ubuntu@trial084.front.sepia.ceph.com 2026-02-22T20:28:56.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T20:28:56.872 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-22T20:28:56.884 DEBUG:teuthology.orchestra.run.trial084:> uname -m 2026-02-22T20:28:56.945 INFO:teuthology.orchestra.run.trial084.stdout:x86_64 2026-02-22T20:28:56.946 DEBUG:teuthology.orchestra.run.trial084:> cat /etc/os-release 2026-02-22T20:28:57.000 INFO:teuthology.orchestra.run.trial084.stdout:NAME="Rocky Linux" 2026-02-22T20:28:57.000 INFO:teuthology.orchestra.run.trial084.stdout:VERSION="10.1 (Red Quartz)" 2026-02-22T20:28:57.000 INFO:teuthology.orchestra.run.trial084.stdout:ID="rocky" 2026-02-22T20:28:57.000 INFO:teuthology.orchestra.run.trial084.stdout:ID_LIKE="rhel centos fedora" 2026-02-22T20:28:57.000 INFO:teuthology.orchestra.run.trial084.stdout:VERSION_ID="10.1" 2026-02-22T20:28:57.000 INFO:teuthology.orchestra.run.trial084.stdout:PLATFORM_ID="platform:el10" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:PRETTY_NAME="Rocky Linux 10.1 (Red Quartz)" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:ANSI_COLOR="0;32" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:LOGO="fedora-logo-icon" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:CPE_NAME="cpe:/o:rocky:rocky:10::baseos" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:HOME_URL="https://rockylinux.org/" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:VENDOR_NAME="RESF" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:VENDOR_URL="https://resf.org/" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:BUG_REPORT_URL="https://bugs.rockylinux.org/" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:SUPPORT_END="2035-05-31" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:ROCKY_SUPPORT_PRODUCT="Rocky-Linux-10" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:ROCKY_SUPPORT_PRODUCT_VERSION="10.1" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:REDHAT_SUPPORT_PRODUCT="Rocky Linux" 2026-02-22T20:28:57.001 INFO:teuthology.orchestra.run.trial084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="10.1" 2026-02-22T20:28:57.002 INFO:teuthology.lock.ops:Updating trial084.front.sepia.ceph.com on lock server 2026-02-22T20:28:57.129 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-22T20:28:57.141 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-22T20:28:57.152 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-22T20:28:57.152 DEBUG:teuthology.orchestra.run.trial084:> test '!' -e /home/ubuntu/cephtest 2026-02-22T20:28:57.169 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-22T20:28:57.182 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-22T20:28:57.182 DEBUG:teuthology.orchestra.run.trial084:> test -z $(ls -A /var/lib/ceph) 2026-02-22T20:28:57.223 INFO:teuthology.orchestra.run.trial084.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-22T20:28:57.224 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-22T20:28:57.318 INFO:teuthology.run_tasks:Running task kernel... 2026-02-22T20:28:57.337 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-22T20:28:57.337 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-22T20:28:57.337 DEBUG:teuthology.orchestra.run.trial084:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-22T20:28:57.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T20:28:57.353 DEBUG:teuthology.orchestra.run.trial084:> uname -r 2026-02-22T20:28:57.410 INFO:teuthology.orchestra.run.trial084.stdout:6.12.0-124.35.1.el10_1.x86_64 2026-02-22T20:28:57.411 INFO:teuthology.task.kernel:Running kernel on trial084: 6.12.0-124.35.1.el10_1.x86_64 2026-02-22T20:28:57.411 DEBUG:teuthology.orchestra.run.trial084:> sudo yum install -y kernel 2026-02-22T20:28:57.931 INFO:teuthology.orchestra.run.trial084.stdout:Rocky Linux 10 - AppStream 23 MB/s | 2.2 MB 00:00 2026-02-22T20:28:58.310 INFO:teuthology.orchestra.run.trial084.stdout:Rocky Linux 10 - BaseOS 48 MB/s | 12 MB 00:00 2026-02-22T20:28:58.855 INFO:teuthology.orchestra.run.trial084.stdout:Rocky Linux 10 - CRB 5.1 MB/s | 508 kB 00:00 2026-02-22T20:29:00.254 INFO:teuthology.orchestra.run.trial084.stdout:Extra Packages for Enterprise Linux 4.6 MB/s | 6.2 MB 00:01 2026-02-22T20:29:00.909 INFO:teuthology.orchestra.run.trial084.stdout:lab-extras 327 kB/s | 3.7 kB 00:00 2026-02-22T20:29:01.315 INFO:teuthology.orchestra.run.trial084.stdout:Package kernel-6.12.0-124.27.1.el10_1.x86_64 is already installed. 2026-02-22T20:29:01.316 INFO:teuthology.orchestra.run.trial084.stdout:Package kernel-6.12.0-124.35.1.el10_1.x86_64 is already installed. 2026-02-22T20:29:01.327 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:29:01.328 INFO:teuthology.orchestra.run.trial084.stdout:Nothing to do. 2026-02-22T20:29:01.328 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:29:01.351 DEBUG:teuthology.orchestra.run.trial084:> echo no | sudo yum reinstall kernel || true 2026-02-22T20:29:01.635 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:01 ago on Sun 22 Feb 2026 08:29:00 PM UTC. 2026-02-22T20:29:01.673 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:29:01.674 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:29:01.674 INFO:teuthology.orchestra.run.trial084.stdout: Package Architecture Version Repository Size 2026-02-22T20:29:01.674 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:29:01.674 INFO:teuthology.orchestra.run.trial084.stdout:Reinstalling: 2026-02-22T20:29:01.674 INFO:teuthology.orchestra.run.trial084.stdout: kernel x86_64 6.12.0-124.27.1.el10_1 baseos 1.4 M 2026-02-22T20:29:01.674 INFO:teuthology.orchestra.run.trial084.stdout: kernel x86_64 6.12.0-124.35.1.el10_1 baseos 1.4 M 2026-02-22T20:29:01.674 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:29:01.674 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:29:01.675 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:29:01.675 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:29:01.675 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 2.8 M 2026-02-22T20:29:01.675 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 0 2026-02-22T20:29:01.675 INFO:teuthology.orchestra.run.trial084.stderr:Operation aborted. 2026-02-22T20:29:01.693 INFO:teuthology.orchestra.run.trial084.stdout:Is this ok [y/N]: 2026-02-22T20:29:01.693 DEBUG:teuthology.orchestra.run.trial084:> sudo yum reinstall -y kernel || true 2026-02-22T20:29:01.976 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:01 ago on Sun 22 Feb 2026 08:29:00 PM UTC. 2026-02-22T20:29:02.014 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:29:02.014 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:29:02.014 INFO:teuthology.orchestra.run.trial084.stdout: Package Architecture Version Repository Size 2026-02-22T20:29:02.015 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:29:02.015 INFO:teuthology.orchestra.run.trial084.stdout:Reinstalling: 2026-02-22T20:29:02.015 INFO:teuthology.orchestra.run.trial084.stdout: kernel x86_64 6.12.0-124.27.1.el10_1 baseos 1.4 M 2026-02-22T20:29:02.015 INFO:teuthology.orchestra.run.trial084.stdout: kernel x86_64 6.12.0-124.35.1.el10_1 baseos 1.4 M 2026-02-22T20:29:02.015 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:29:02.015 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:29:02.015 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:29:02.015 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:29:02.015 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 2.8 M 2026-02-22T20:29:02.015 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 0 2026-02-22T20:29:02.015 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:29:02.069 INFO:teuthology.orchestra.run.trial084.stdout:(1/2): kernel-6.12.0-124.35.1.el10_1.x86_64.rpm 26 MB/s | 1.4 MB 00:00 2026-02-22T20:29:02.071 INFO:teuthology.orchestra.run.trial084.stdout:(2/2): kernel-6.12.0-124.27.1.el10_1.x86_64.rpm 24 MB/s | 1.4 MB 00:00 2026-02-22T20:29:02.072 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:29:02.072 INFO:teuthology.orchestra.run.trial084.stdout:Total 48 MB/s | 2.8 MB 00:00 2026-02-22T20:29:02.072 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:29:02.151 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:29:02.151 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:29:02.172 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:29:02.172 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:29:02.216 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:29:02.224 INFO:teuthology.orchestra.run.trial084.stdout: Reinstalling : kernel-6.12.0-124.35.1.el10_1.x86_64 1/4 2026-02-22T20:29:02.227 INFO:teuthology.orchestra.run.trial084.stdout: Reinstalling : kernel-6.12.0-124.27.1.el10_1.x86_64 2/4 2026-02-22T20:29:02.228 INFO:teuthology.orchestra.run.trial084.stdout: Cleanup : kernel-6.12.0-124.35.1.el10_1.x86_64 3/4 2026-02-22T20:29:02.340 INFO:teuthology.orchestra.run.trial084.stdout: Cleanup : kernel-6.12.0-124.27.1.el10_1.x86_64 4/4 2026-02-22T20:29:02.340 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:29:02.340 INFO:teuthology.orchestra.run.trial084.stdout:Reinstalled: 2026-02-22T20:29:02.341 INFO:teuthology.orchestra.run.trial084.stdout: kernel-6.12.0-124.27.1.el10_1.x86_64 kernel-6.12.0-124.35.1.el10_1.x86_64 2026-02-22T20:29:02.341 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:29:02.341 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:29:02.366 DEBUG:teuthology.orchestra.run.trial084:> rpm -q kernel | sort -rV | head -n 1 2026-02-22T20:29:02.399 INFO:teuthology.orchestra.run.trial084.stdout:kernel-6.12.0-124.35.1.el10_1.x86_64 2026-02-22T20:29:02.399 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 6.12.0-124.35.1.el10_1.x86_64 2026-02-22T20:29:02.400 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-22T20:29:02.400 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-22T20:29:02.400 DEBUG:teuthology.orchestra.run.trial084:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-22T20:29:02.465 INFO:teuthology.orchestra.run.trial084.stdout:ttyS1 2026-02-22T20:29:02.500 DEBUG:teuthology.parallel:result is None 2026-02-22T20:29:02.500 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-22T20:29:02.515 INFO:teuthology.task.internal:Creating test directory... 2026-02-22T20:29:02.516 DEBUG:teuthology.orchestra.run.trial084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-22T20:29:02.557 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-22T20:29:02.568 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-22T20:29:02.581 INFO:teuthology.task.internal:Creating archive directory... 2026-02-22T20:29:02.581 DEBUG:teuthology.orchestra.run.trial084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-22T20:29:02.619 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-22T20:29:02.631 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-22T20:29:02.631 DEBUG:teuthology.orchestra.run.trial084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-22T20:29:02.682 INFO:teuthology.orchestra.run.trial084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T20:29:02.692 INFO:teuthology.orchestra.run.trial084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T20:29:02.694 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-22T20:29:02.705 INFO:teuthology.task.internal:Configuring sudo... 2026-02-22T20:29:02.705 DEBUG:teuthology.orchestra.run.trial084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-22T20:29:02.760 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-22T20:29:02.773 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-22T20:29:02.773 DEBUG:teuthology.orchestra.run.trial084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-22T20:29:02.817 DEBUG:teuthology.orchestra.run.trial084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T20:29:02.971 DEBUG:teuthology.orchestra.run.trial084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T20:29:02.997 DEBUG:teuthology.orchestra.run.trial084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T20:29:03.139 DEBUG:teuthology.orchestra.run.trial084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T20:29:03.165 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:29:03.165 DEBUG:teuthology.orchestra.run.trial084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-22T20:29:03.232 DEBUG:teuthology.orchestra.run.trial084:> sudo service rsyslog restart 2026-02-22T20:29:03.303 INFO:teuthology.orchestra.run.trial084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-22T20:29:03.752 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-22T20:29:03.768 INFO:teuthology.task.internal:Starting timer... 2026-02-22T20:29:03.768 INFO:teuthology.run_tasks:Running task pcp... 2026-02-22T20:29:03.789 INFO:teuthology.run_tasks:Running task selinux... 2026-02-22T20:29:03.807 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['comm="systemd".*denied.*\\{ prog_run \\}.*tclass=bpf.*permissive=1']} 2026-02-22T20:29:03.807 DEBUG:teuthology.orchestra.run.trial084:> sudo service auditd rotate 2026-02-22T20:29:03.837 INFO:teuthology.orchestra.run.trial084.stdout:Rotating logs: 2026-02-22T20:29:03.839 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-22T20:29:03.839 DEBUG:teuthology.orchestra.run.trial084:> /usr/sbin/getenforce 2026-02-22T20:29:03.896 INFO:teuthology.orchestra.run.trial084.stdout:Permissive 2026-02-22T20:29:03.896 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial084.front.sepia.ceph.com': 'permissive'} 2026-02-22T20:29:03.896 DEBUG:teuthology.orchestra.run.trial084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e 'comm="systemd".*denied.*\{ prog_run \}.*tclass=bpf.*permissive=1' 2026-02-22T20:29:03.953 INFO:teuthology.orchestra.run.trial084.stderr:grep: Invalid content of \{\} 2026-02-22T20:29:03.962 DEBUG:teuthology.orchestra.run:got remote process result: 2 2026-02-22T20:29:03.962 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-22T20:29:03.962 DEBUG:teuthology.orchestra.run.trial084:> sudo /usr/sbin/setenforce permissive 2026-02-22T20:29:04.028 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-22T20:29:04.041 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-22T20:29:04.050 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-22T20:29:04.051 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial084.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-22T20:31:44.673 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial084.front.sepia.ceph.com')] 2026-02-22T20:31:44.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial084.front.sepia.ceph.com' 2026-02-22T20:31:44.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T20:31:44.756 DEBUG:teuthology.orchestra.run.trial084:> true 2026-02-22T20:31:44.834 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial084.front.sepia.ceph.com' 2026-02-22T20:31:44.835 INFO:teuthology.run_tasks:Running task clock... 2026-02-22T20:31:44.848 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-22T20:31:44.849 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-22T20:31:44.849 DEBUG:teuthology.orchestra.run.trial084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T20:31:44.905 INFO:teuthology.orchestra.run.trial084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-22T20:31:44.918 INFO:teuthology.orchestra.run.trial084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-22T20:31:44.940 INFO:teuthology.orchestra.run.trial084.stderr:sudo: ntpd: command not found 2026-02-22T20:31:44.951 INFO:teuthology.orchestra.run.trial084.stdout:506 Cannot talk to daemon 2026-02-22T20:31:44.963 INFO:teuthology.orchestra.run.trial084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-22T20:31:44.976 INFO:teuthology.orchestra.run.trial084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-22T20:31:45.031 INFO:teuthology.orchestra.run.trial084.stderr:bash: line 1: ntpq: command not found 2026-02-22T20:31:45.033 INFO:teuthology.orchestra.run.trial084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-22T20:31:45.033 INFO:teuthology.orchestra.run.trial084.stdout:=============================================================================== 2026-02-22T20:31:45.033 INFO:teuthology.orchestra.run.trial084.stdout:^? ntp.nyc.icanbwell.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T20:31:45.033 INFO:teuthology.orchestra.run.trial084.stdout:^? 144.202.41.38.vultruserc> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T20:31:45.033 INFO:teuthology.orchestra.run.trial084.stdout:^? ntp11.kernfusion.at 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T20:31:45.033 INFO:teuthology.orchestra.run.trial084.stdout:^? ntp.maxhost.io 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T20:31:45.034 INFO:teuthology.run_tasks:Running task install... 2026-02-22T20:31:45.046 DEBUG:teuthology.task.install:project ceph 2026-02-22T20:31:45.046 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a5c3084f688b1f6eff223b8d259a8316f9844c8e'}} 2026-02-22T20:31:45.046 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'a5c3084f688b1f6eff223b8d259a8316f9844c8e'} 2026-02-22T20:31:45.046 INFO:teuthology.task.install:Using flavor: default 2026-02-22T20:31:45.049 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-22T20:31:45.049 INFO:teuthology.task.install:extra packages: [] 2026-02-22T20:31:45.049 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a5c3084f688b1f6eff223b8d259a8316f9844c8e', 'tag': None, 'wait_for_package': False} 2026-02-22T20:31:45.049 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=rocky%2F10%2Fx86_64&sha1=a5c3084f688b1f6eff223b8d259a8316f9844c8e 2026-02-22T20:31:45.169 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle/a5c3084f688b1f6eff223b8d259a8316f9844c8e/rocky/10/flavors/default/ 2026-02-22T20:31:45.169 INFO:teuthology.task.install.rpm:Package version is 20.2.0-794.ga5c3084f 2026-02-22T20:31:45.292 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle/a5c3084f688b1f6eff223b8d259a8316f9844c8e/rocky/10/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle/a5c3084f688b1f6eff223b8d259a8316f9844c8e/rocky/10/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle/a5c3084f688b1f6eff223b8d259a8316f9844c8e/rocky/10/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-22T20:31:45.292 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:31:45.292 DEBUG:teuthology.orchestra.run.trial084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-22T20:31:45.321 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-22T20:31:45.321 DEBUG:teuthology.orchestra.run.trial084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a5c3084f688b1f6eff223b8d259a8316f9844c8e/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-22T20:31:45.389 DEBUG:teuthology.orchestra.run.trial084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-22T20:31:45.467 DEBUG:teuthology.orchestra.run.trial084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-22T20:31:45.534 INFO:teuthology.orchestra.run.trial084.stdout:check_obsoletes = 1 2026-02-22T20:31:45.535 DEBUG:teuthology.orchestra.run.trial084:> sudo yum clean all 2026-02-22T20:31:45.687 INFO:teuthology.orchestra.run.trial084.stdout:28 files removed 2026-02-22T20:31:45.702 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install ceph-radosgw 2026-02-22T20:31:45.964 INFO:teuthology.orchestra.run.trial084.stdout:ceph packages for x86_64 227 kB/s | 24 kB 00:00 2026-02-22T20:31:46.067 INFO:teuthology.orchestra.run.trial084.stdout:ceph noarch packages 41 kB/s | 3.8 kB 00:00 2026-02-22T20:31:46.170 INFO:teuthology.orchestra.run.trial084.stdout:ceph source packages 17 kB/s | 1.7 kB 00:00 2026-02-22T20:31:46.312 INFO:teuthology.orchestra.run.trial084.stdout:Rocky Linux 10 - AppStream 16 MB/s | 2.2 MB 00:00 2026-02-22T20:31:46.649 INFO:teuthology.orchestra.run.trial084.stdout:Rocky Linux 10 - BaseOS 58 MB/s | 12 MB 00:00 2026-02-22T20:31:47.173 INFO:teuthology.orchestra.run.trial084.stdout:Rocky Linux 10 - CRB 6.4 MB/s | 508 kB 00:00 2026-02-22T20:31:48.194 INFO:teuthology.orchestra.run.trial084.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 6.2 MB 00:00 2026-02-22T20:31:48.844 INFO:teuthology.orchestra.run.trial084.stdout:lab-extras 207 kB/s | 3.7 kB 00:00 2026-02-22T20:31:49.367 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:31:49.368 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:31:49.368 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repository Size 2026-02-22T20:31:49.368 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:31:49.368 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:31:49.368 INFO:teuthology.orchestra.run.trial084.stdout: ceph-radosgw x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 24 M 2026-02-22T20:31:49.369 INFO:teuthology.orchestra.run.trial084.stdout:Upgrading: 2026-02-22T20:31:49.369 INFO:teuthology.orchestra.run.trial084.stdout: librados2 x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 3.7 M 2026-02-22T20:31:49.369 INFO:teuthology.orchestra.run.trial084.stdout: librbd1 x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 2.9 M 2026-02-22T20:31:49.369 INFO:teuthology.orchestra.run.trial084.stdout:Installing dependencies: 2026-02-22T20:31:49.369 INFO:teuthology.orchestra.run.trial084.stdout: abseil-cpp x86_64 20240722.1-1.el10_0 epel 590 k 2026-02-22T20:31:49.369 INFO:teuthology.orchestra.run.trial084.stdout: boost-program-options x86_64 1.83.0-5.el10 appstream 109 k 2026-02-22T20:31:49.369 INFO:teuthology.orchestra.run.trial084.stdout: ceph-base x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 6.0 M 2026-02-22T20:31:49.369 INFO:teuthology.orchestra.run.trial084.stdout: ceph-common x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 24 M 2026-02-22T20:31:49.369 INFO:teuthology.orchestra.run.trial084.stdout: ceph-selinux x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 25 k 2026-02-22T20:31:49.369 INFO:teuthology.orchestra.run.trial084.stdout: daxctl-libs x86_64 80-4.el10 baseos 42 k 2026-02-22T20:31:49.369 INFO:teuthology.orchestra.run.trial084.stdout: gperftools-libs x86_64 2.15-4.el10_0 epel 303 k 2026-02-22T20:31:49.369 INFO:teuthology.orchestra.run.trial084.stdout: libarrow x86_64 17.0.0-9.el10_2 epel 5.7 M 2026-02-22T20:31:49.370 INFO:teuthology.orchestra.run.trial084.stdout: libarrow-doc noarch 17.0.0-9.el10_2 epel 27 k 2026-02-22T20:31:49.370 INFO:teuthology.orchestra.run.trial084.stdout: libcephfs2 x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 897 k 2026-02-22T20:31:49.370 INFO:teuthology.orchestra.run.trial084.stdout: libnbd x86_64 1.22.2-3.el10_1 appstream 175 k 2026-02-22T20:31:49.370 INFO:teuthology.orchestra.run.trial084.stdout: liboath x86_64 2.6.12-1.el10_0 epel 48 k 2026-02-22T20:31:49.370 INFO:teuthology.orchestra.run.trial084.stdout: liborc2 x86_64 2.0.7-1.el10_2 epel 515 k 2026-02-22T20:31:49.370 INFO:teuthology.orchestra.run.trial084.stdout: libpmem x86_64 2.1.0-2.el10_0 epel 112 k 2026-02-22T20:31:49.370 INFO:teuthology.orchestra.run.trial084.stdout: libpmemobj x86_64 2.1.0-2.el10_0 epel 161 k 2026-02-22T20:31:49.370 INFO:teuthology.orchestra.run.trial084.stdout: librabbitmq x86_64 0.14.0-3.el10 appstream 43 k 2026-02-22T20:31:49.370 INFO:teuthology.orchestra.run.trial084.stdout: libradosstriper1 x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 254 k 2026-02-22T20:31:49.370 INFO:teuthology.orchestra.run.trial084.stdout: librdkafka x86_64 2.1.1-7.el10 appstream 725 k 2026-02-22T20:31:49.370 INFO:teuthology.orchestra.run.trial084.stdout: librgw2 x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 6.6 M 2026-02-22T20:31:49.370 INFO:teuthology.orchestra.run.trial084.stdout: libunwind x86_64 1.8.0-4.el10_0 epel 73 k 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: lttng-ust x86_64 2.13.7-5.el10 crb 334 k 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: mailcap noarch 2.1.54-8.el10 baseos 34 k 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: ndctl-libs x86_64 80-4.el10 baseos 89 k 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: parquet-libs x86_64 17.0.0-9.el10_2 epel 1.0 M 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: pciutils x86_64 3.13.0-6.el10 baseos 122 k 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: protobuf x86_64 3.19.6-11.el10 appstream 1.0 M 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: python3-ceph-argparse x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 57 k 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: python3-ceph-common x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 232 k 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: python3-cephfs x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 206 k 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: python3-prettytable noarch 3.6.0-10.el10 appstream 70 k 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: python3-rados x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 350 k 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: python3-rbd x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 367 k 2026-02-22T20:31:49.371 INFO:teuthology.orchestra.run.trial084.stdout: python3-rgw x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 130 k 2026-02-22T20:31:49.372 INFO:teuthology.orchestra.run.trial084.stdout: python3-wcwidth noarch 0.2.6-6.el10 baseos 48 k 2026-02-22T20:31:49.372 INFO:teuthology.orchestra.run.trial084.stdout: qatlib x86_64 24.09.0-2.el10 appstream 229 k 2026-02-22T20:31:49.372 INFO:teuthology.orchestra.run.trial084.stdout: qatzip-libs x86_64 1.3.1-1.el10 appstream 66 k 2026-02-22T20:31:49.372 INFO:teuthology.orchestra.run.trial084.stdout: re2 x86_64 2:20251105-1.el10_2 epel 221 k 2026-02-22T20:31:49.372 INFO:teuthology.orchestra.run.trial084.stdout: thrift x86_64 0.20.0-4.el10_1 epel 1.7 M 2026-02-22T20:31:49.372 INFO:teuthology.orchestra.run.trial084.stdout: utf8proc x86_64 2.7.0-9.el10 appstream 80 k 2026-02-22T20:31:49.372 INFO:teuthology.orchestra.run.trial084.stdout:Installing weak dependencies: 2026-02-22T20:31:49.372 INFO:teuthology.orchestra.run.trial084.stdout: qatlib-service x86_64 24.09.0-2.el10 appstream 37 k 2026-02-22T20:31:49.372 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:31:49.372 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:31:49.372 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:31:49.372 INFO:teuthology.orchestra.run.trial084.stdout:Install 41 Packages 2026-02-22T20:31:49.373 INFO:teuthology.orchestra.run.trial084.stdout:Upgrade 2 Packages 2026-02-22T20:31:49.373 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:31:49.373 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 84 M 2026-02-22T20:31:49.373 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:31:50.993 INFO:teuthology.orchestra.run.trial084.stdout:(1/43): ceph-base-20.2.0-794.ga5c3084f.el10.x86 4.0 MB/s | 6.0 MB 00:01 2026-02-22T20:31:51.055 INFO:teuthology.orchestra.run.trial084.stdout:(2/43): ceph-selinux-20.2.0-794.ga5c3084f.el10. 409 kB/s | 25 kB 00:00 2026-02-22T20:31:51.197 INFO:teuthology.orchestra.run.trial084.stdout:(3/43): libcephfs2-20.2.0-794.ga5c3084f.el10.x8 6.2 MB/s | 897 kB 00:00 2026-02-22T20:31:51.243 INFO:teuthology.orchestra.run.trial084.stdout:(4/43): libradosstriper1-20.2.0-794.ga5c3084f.e 5.4 MB/s | 254 kB 00:00 2026-02-22T20:31:52.329 INFO:teuthology.orchestra.run.trial084.stdout:(5/43): librgw2-20.2.0-794.ga5c3084f.el10.x86_6 6.1 MB/s | 6.6 MB 00:01 2026-02-22T20:31:52.392 INFO:teuthology.orchestra.run.trial084.stdout:(6/43): python3-ceph-argparse-20.2.0-794.ga5c30 909 kB/s | 57 kB 00:00 2026-02-22T20:31:52.459 INFO:teuthology.orchestra.run.trial084.stdout:(7/43): python3-ceph-common-20.2.0-794.ga5c3084 3.4 MB/s | 232 kB 00:00 2026-02-22T20:31:52.527 INFO:teuthology.orchestra.run.trial084.stdout:(8/43): python3-cephfs-20.2.0-794.ga5c3084f.el1 3.0 MB/s | 206 kB 00:00 2026-02-22T20:31:52.597 INFO:teuthology.orchestra.run.trial084.stdout:(9/43): python3-rados-20.2.0-794.ga5c3084f.el10 4.8 MB/s | 350 kB 00:00 2026-02-22T20:31:52.669 INFO:teuthology.orchestra.run.trial084.stdout:(10/43): python3-rbd-20.2.0-794.ga5c3084f.el10. 5.0 MB/s | 367 kB 00:00 2026-02-22T20:31:52.736 INFO:teuthology.orchestra.run.trial084.stdout:(11/43): python3-rgw-20.2.0-794.ga5c3084f.el10. 1.9 MB/s | 130 kB 00:00 2026-02-22T20:31:52.776 INFO:teuthology.orchestra.run.trial084.stdout:(12/43): boost-program-options-1.83.0-5.el10.x8 2.7 MB/s | 109 kB 00:00 2026-02-22T20:31:52.796 INFO:teuthology.orchestra.run.trial084.stdout:(13/43): libnbd-1.22.2-3.el10_1.x86_64.rpm 8.6 MB/s | 175 kB 00:00 2026-02-22T20:31:52.811 INFO:teuthology.orchestra.run.trial084.stdout:(14/43): librabbitmq-0.14.0-3.el10.x86_64.rpm 2.8 MB/s | 43 kB 00:00 2026-02-22T20:31:52.840 INFO:teuthology.orchestra.run.trial084.stdout:(15/43): librdkafka-2.1.1-7.el10.x86_64.rpm 25 MB/s | 725 kB 00:00 2026-02-22T20:31:52.868 INFO:teuthology.orchestra.run.trial084.stdout:(16/43): protobuf-3.19.6-11.el10.x86_64.rpm 36 MB/s | 1.0 MB 00:00 2026-02-22T20:31:52.887 INFO:teuthology.orchestra.run.trial084.stdout:(17/43): python3-prettytable-3.6.0-10.el10.noar 3.7 MB/s | 70 kB 00:00 2026-02-22T20:31:52.904 INFO:teuthology.orchestra.run.trial084.stdout:(18/43): qatlib-24.09.0-2.el10.x86_64.rpm 13 MB/s | 229 kB 00:00 2026-02-22T20:31:52.923 INFO:teuthology.orchestra.run.trial084.stdout:(19/43): qatlib-service-24.09.0-2.el10.x86_64.r 1.9 MB/s | 37 kB 00:00 2026-02-22T20:31:52.940 INFO:teuthology.orchestra.run.trial084.stdout:(20/43): qatzip-libs-1.3.1-1.el10.x86_64.rpm 3.9 MB/s | 66 kB 00:00 2026-02-22T20:31:52.957 INFO:teuthology.orchestra.run.trial084.stdout:(21/43): utf8proc-2.7.0-9.el10.x86_64.rpm 4.7 MB/s | 80 kB 00:00 2026-02-22T20:31:52.971 INFO:teuthology.orchestra.run.trial084.stdout:(22/43): daxctl-libs-80-4.el10.x86_64.rpm 2.9 MB/s | 42 kB 00:00 2026-02-22T20:31:52.990 INFO:teuthology.orchestra.run.trial084.stdout:(23/43): mailcap-2.1.54-8.el10.noarch.rpm 1.8 MB/s | 34 kB 00:00 2026-02-22T20:31:53.007 INFO:teuthology.orchestra.run.trial084.stdout:(24/43): ndctl-libs-80-4.el10.x86_64.rpm 5.1 MB/s | 89 kB 00:00 2026-02-22T20:31:53.023 INFO:teuthology.orchestra.run.trial084.stdout:(25/43): pciutils-3.13.0-6.el10.x86_64.rpm 7.5 MB/s | 122 kB 00:00 2026-02-22T20:31:53.036 INFO:teuthology.orchestra.run.trial084.stdout:(26/43): python3-wcwidth-0.2.6-6.el10.noarch.rp 3.6 MB/s | 48 kB 00:00 2026-02-22T20:31:53.054 INFO:teuthology.orchestra.run.trial084.stdout:(27/43): lttng-ust-2.13.7-5.el10.x86_64.rpm 18 MB/s | 334 kB 00:00 2026-02-22T20:31:53.533 INFO:teuthology.orchestra.run.trial084.stdout:(28/43): abseil-cpp-20240722.1-1.el10_0.x86_64. 1.2 MB/s | 590 kB 00:00 2026-02-22T20:31:53.616 INFO:teuthology.orchestra.run.trial084.stdout:(29/43): gperftools-libs-2.15-4.el10_0.x86_64.r 3.6 MB/s | 303 kB 00:00 2026-02-22T20:31:53.895 INFO:teuthology.orchestra.run.trial084.stdout:(30/43): libarrow-17.0.0-9.el10_2.x86_64.rpm 20 MB/s | 5.7 MB 00:00 2026-02-22T20:31:53.958 INFO:teuthology.orchestra.run.trial084.stdout:(31/43): libarrow-doc-17.0.0-9.el10_2.noarch.rp 427 kB/s | 27 kB 00:00 2026-02-22T20:31:54.020 INFO:teuthology.orchestra.run.trial084.stdout:(32/43): liboath-2.6.12-1.el10_0.x86_64.rpm 772 kB/s | 48 kB 00:00 2026-02-22T20:31:54.086 INFO:teuthology.orchestra.run.trial084.stdout:(33/43): liborc2-2.0.7-1.el10_2.x86_64.rpm 7.7 MB/s | 515 kB 00:00 2026-02-22T20:31:54.151 INFO:teuthology.orchestra.run.trial084.stdout:(34/43): libpmem-2.1.0-2.el10_0.x86_64.rpm 1.7 MB/s | 112 kB 00:00 2026-02-22T20:31:54.215 INFO:teuthology.orchestra.run.trial084.stdout:(35/43): libpmemobj-2.1.0-2.el10_0.x86_64.rpm 2.5 MB/s | 161 kB 00:00 2026-02-22T20:31:54.278 INFO:teuthology.orchestra.run.trial084.stdout:(36/43): libunwind-1.8.0-4.el10_0.x86_64.rpm 1.1 MB/s | 73 kB 00:00 2026-02-22T20:31:54.349 INFO:teuthology.orchestra.run.trial084.stdout:(37/43): parquet-libs-17.0.0-9.el10_2.x86_64.rp 14 MB/s | 1.0 MB 00:00 2026-02-22T20:31:54.413 INFO:teuthology.orchestra.run.trial084.stdout:(38/43): re2-20251105-1.el10_2.x86_64.rpm 3.4 MB/s | 221 kB 00:00 2026-02-22T20:31:54.489 INFO:teuthology.orchestra.run.trial084.stdout:(39/43): thrift-0.20.0-4.el10_1.x86_64.rpm 22 MB/s | 1.7 MB 00:00 2026-02-22T20:31:55.326 INFO:teuthology.orchestra.run.trial084.stdout:(40/43): librados2-20.2.0-794.ga5c3084f.el10.x8 4.4 MB/s | 3.7 MB 00:00 2026-02-22T20:31:55.732 INFO:teuthology.orchestra.run.trial084.stdout:(41/43): librbd1-20.2.0-794.ga5c3084f.el10.x86_ 7.0 MB/s | 2.9 MB 00:00 2026-02-22T20:31:57.951 INFO:teuthology.orchestra.run.trial084.stdout:(42/43): ceph-common-20.2.0-794.ga5c3084f.el10. 2.9 MB/s | 24 MB 00:08 2026-02-22T20:32:02.902 INFO:teuthology.orchestra.run.trial084.stdout:(43/43): ceph-radosgw-20.2.0-794.ga5c3084f.el10 1.8 MB/s | 24 MB 00:13 2026-02-22T20:32:02.903 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:02.903 INFO:teuthology.orchestra.run.trial084.stdout:Total 6.2 MB/s | 84 MB 00:13 2026-02-22T20:32:02.903 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:02.923 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:02.924 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:03.241 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:03.241 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:03.598 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:03.602 INFO:teuthology.orchestra.run.trial084.stdout: Installing : lttng-ust-2.13.7-5.el10.x86_64 1/45 2026-02-22T20:32:03.608 INFO:teuthology.orchestra.run.trial084.stdout: Installing : daxctl-libs-80-4.el10.x86_64 2/45 2026-02-22T20:32:03.610 INFO:teuthology.orchestra.run.trial084.stdout: Installing : librdkafka-2.1.1-7.el10.x86_64 3/45 2026-02-22T20:32:03.612 INFO:teuthology.orchestra.run.trial084.stdout: Installing : librabbitmq-0.14.0-3.el10.x86_64 4/45 2026-02-22T20:32:03.623 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ndctl-libs-80-4.el10.x86_64 5/45 2026-02-22T20:32:03.739 INFO:teuthology.orchestra.run.trial084.stdout: Installing : thrift-0.20.0-4.el10_1.x86_64 6/45 2026-02-22T20:32:03.743 INFO:teuthology.orchestra.run.trial084.stdout: Upgrading : librados2-2:20.2.0-794.ga5c3084f.el10.x86_64 7/45 2026-02-22T20:32:03.768 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: librados2-2:20.2.0-794.ga5c3084f.el10.x86_64 7/45 2026-02-22T20:32:03.797 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-rados-2:20.2.0-794.ga5c3084f.el10.x86_64 8/45 2026-02-22T20:32:03.798 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libcephfs2-2:20.2.0-794.ga5c3084f.el10.x86_64 9/45 2026-02-22T20:32:03.809 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: libcephfs2-2:20.2.0-794.ga5c3084f.el10.x86_64 9/45 2026-02-22T20:32:03.811 INFO:teuthology.orchestra.run.trial084.stdout: Installing : liboath-2.6.12-1.el10_0.x86_64 10/45 2026-02-22T20:32:03.814 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libarrow-doc-17.0.0-9.el10_2.noarch 11/45 2026-02-22T20:32:03.822 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-ceph-argparse-2:20.2.0-794.ga5c3084f.el10. 12/45 2026-02-22T20:32:03.830 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-cephfs-2:20.2.0-794.ga5c3084f.el10.x86_64 13/45 2026-02-22T20:32:03.831 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libradosstriper1-2:20.2.0-794.ga5c3084f.el10.x86_6 14/45 2026-02-22T20:32:03.843 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: libradosstriper1-2:20.2.0-794.ga5c3084f.el10.x86_6 14/45 2026-02-22T20:32:03.846 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libpmem-2.1.0-2.el10_0.x86_64 15/45 2026-02-22T20:32:03.850 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libpmemobj-2.1.0-2.el10_0.x86_64 16/45 2026-02-22T20:32:03.855 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libunwind-1.8.0-4.el10_0.x86_64 17/45 2026-02-22T20:32:03.881 INFO:teuthology.orchestra.run.trial084.stdout: Installing : gperftools-libs-2.15-4.el10_0.x86_64 18/45 2026-02-22T20:32:03.885 INFO:teuthology.orchestra.run.trial084.stdout: Installing : abseil-cpp-20240722.1-1.el10_0.x86_64 19/45 2026-02-22T20:32:03.889 INFO:teuthology.orchestra.run.trial084.stdout: Installing : re2-2:20251105-1.el10_2.x86_64 20/45 2026-02-22T20:32:03.891 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-wcwidth-0.2.6-6.el10.noarch 21/45 2026-02-22T20:32:03.895 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-prettytable-3.6.0-10.el10.noarch 22/45 2026-02-22T20:32:03.897 INFO:teuthology.orchestra.run.trial084.stdout: Installing : pciutils-3.13.0-6.el10.x86_64 23/45 2026-02-22T20:32:03.898 INFO:teuthology.orchestra.run.trial084.stdout: Installing : qatlib-service-24.09.0-2.el10.x86_64 24/45 2026-02-22T20:32:03.913 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: qatlib-service-24.09.0-2.el10.x86_64 24/45 2026-02-22T20:32:03.929 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: qatlib-24.09.0-2.el10.x86_64 25/45 2026-02-22T20:32:03.931 INFO:teuthology.orchestra.run.trial084.stdout: Installing : qatlib-24.09.0-2.el10.x86_64 25/45 2026-02-22T20:32:03.933 INFO:teuthology.orchestra.run.trial084.stdout: Installing : qatzip-libs-1.3.1-1.el10.x86_64 26/45 2026-02-22T20:32:03.935 INFO:teuthology.orchestra.run.trial084.stdout: Installing : mailcap-2.1.54-8.el10.noarch 27/45 2026-02-22T20:32:03.943 INFO:teuthology.orchestra.run.trial084.stdout: Installing : utf8proc-2.7.0-9.el10.x86_64 28/45 2026-02-22T20:32:03.947 INFO:teuthology.orchestra.run.trial084.stdout: Installing : protobuf-3.19.6-11.el10.x86_64 29/45 2026-02-22T20:32:03.984 INFO:teuthology.orchestra.run.trial084.stdout: Installing : liborc2-2.0.7-1.el10_2.x86_64 30/45 2026-02-22T20:32:03.988 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libarrow-17.0.0-9.el10_2.x86_64 31/45 2026-02-22T20:32:04.077 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libnbd-1.22.2-3.el10_1.x86_64 32/45 2026-02-22T20:32:04.078 INFO:teuthology.orchestra.run.trial084.stdout: Upgrading : librbd1-2:20.2.0-794.ga5c3084f.el10.x86_64 33/45 2026-02-22T20:32:04.103 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: librbd1-2:20.2.0-794.ga5c3084f.el10.x86_64 33/45 2026-02-22T20:32:04.104 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-rbd-2:20.2.0-794.ga5c3084f.el10.x86_64 34/45 2026-02-22T20:32:04.111 INFO:teuthology.orchestra.run.trial084.stdout: Installing : boost-program-options-1.83.0-5.el10.x86_64 35/45 2026-02-22T20:32:04.321 INFO:teuthology.orchestra.run.trial084.stdout: Installing : parquet-libs-17.0.0-9.el10_2.x86_64 36/45 2026-02-22T20:32:04.322 INFO:teuthology.orchestra.run.trial084.stdout: Installing : librgw2-2:20.2.0-794.ga5c3084f.el10.x86_64 37/45 2026-02-22T20:32:04.336 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: librgw2-2:20.2.0-794.ga5c3084f.el10.x86_64 37/45 2026-02-22T20:32:04.352 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-rgw-2:20.2.0-794.ga5c3084f.el10.x86_64 38/45 2026-02-22T20:32:04.354 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-ceph-common-2:20.2.0-794.ga5c3084f.el10.x8 39/45 2026-02-22T20:32:05.136 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-common-2:20.2.0-794.ga5c3084f.el10.x86_64 40/45 2026-02-22T20:32:05.139 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-common-2:20.2.0-794.ga5c3084f.el10.x86_64 40/45 2026-02-22T20:32:05.364 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-common-2:20.2.0-794.ga5c3084f.el10.x86_64 40/45 2026-02-22T20:32:05.367 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-base-2:20.2.0-794.ga5c3084f.el10.x86_64 41/45 2026-02-22T20:32:05.396 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-base-2:20.2.0-794.ga5c3084f.el10.x86_64 41/45 2026-02-22T20:32:05.396 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph.target' → '/usr/lib/systemd/system/ceph.target'. 2026-02-22T20:32:05.396 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-crash.service' → '/usr/lib/systemd/system/ceph-crash.service'. 2026-02-22T20:32:05.396 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:05.399 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-selinux-2:20.2.0-794.ga5c3084f.el10.x86_64 42/45 2026-02-22T20:32:09.340 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-selinux-2:20.2.0-794.ga5c3084f.el10.x86_64 42/45 2026-02-22T20:32:09.340 INFO:teuthology.orchestra.run.trial084.stdout:skipping the directory /sys 2026-02-22T20:32:09.340 INFO:teuthology.orchestra.run.trial084.stdout:skipping the directory /proc 2026-02-22T20:32:09.340 INFO:teuthology.orchestra.run.trial084.stdout:skipping the directory /mnt 2026-02-22T20:32:09.340 INFO:teuthology.orchestra.run.trial084.stdout:skipping the directory /var/tmp 2026-02-22T20:32:09.340 INFO:teuthology.orchestra.run.trial084.stdout:skipping the directory /home 2026-02-22T20:32:09.340 INFO:teuthology.orchestra.run.trial084.stdout:skipping the directory /root 2026-02-22T20:32:09.340 INFO:teuthology.orchestra.run.trial084.stdout:skipping the directory /tmp 2026-02-22T20:32:09.340 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:10.100 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-radosgw-2:20.2.0-794.ga5c3084f.el10.x86_64 43/45 2026-02-22T20:32:10.123 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-radosgw-2:20.2.0-794.ga5c3084f.el10.x86_64 43/45 2026-02-22T20:32:10.123 INFO:teuthology.orchestra.run.trial084.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-22T20:32:10.123 INFO:teuthology.orchestra.run.trial084.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-22T20:32:10.123 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph-radosgw.target' → '/usr/lib/systemd/system/ceph-radosgw.target'. 2026-02-22T20:32:10.123 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-radosgw.target' → '/usr/lib/systemd/system/ceph-radosgw.target'. 2026-02-22T20:32:10.123 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:10.126 INFO:teuthology.orchestra.run.trial084.stdout: Cleanup : librbd1-2:18.2.1-4.el10.x86_64 44/45 2026-02-22T20:32:10.144 INFO:teuthology.orchestra.run.trial084.stdout: Cleanup : librados2-2:18.2.1-4.el10.x86_64 45/45 2026-02-22T20:32:10.616 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: librados2-2:18.2.1-4.el10.x86_64 45/45 2026-02-22T20:32:10.616 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:10.616 INFO:teuthology.orchestra.run.trial084.stdout:Upgraded: 2026-02-22T20:32:10.616 INFO:teuthology.orchestra.run.trial084.stdout: librados2-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.617 INFO:teuthology.orchestra.run.trial084.stdout: librbd1-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.617 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:10.617 INFO:teuthology.orchestra.run.trial084.stdout: abseil-cpp-20240722.1-1.el10_0.x86_64 2026-02-22T20:32:10.617 INFO:teuthology.orchestra.run.trial084.stdout: boost-program-options-1.83.0-5.el10.x86_64 2026-02-22T20:32:10.617 INFO:teuthology.orchestra.run.trial084.stdout: ceph-base-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.617 INFO:teuthology.orchestra.run.trial084.stdout: ceph-common-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.617 INFO:teuthology.orchestra.run.trial084.stdout: ceph-radosgw-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.617 INFO:teuthology.orchestra.run.trial084.stdout: ceph-selinux-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.617 INFO:teuthology.orchestra.run.trial084.stdout: daxctl-libs-80-4.el10.x86_64 2026-02-22T20:32:10.617 INFO:teuthology.orchestra.run.trial084.stdout: gperftools-libs-2.15-4.el10_0.x86_64 2026-02-22T20:32:10.617 INFO:teuthology.orchestra.run.trial084.stdout: libarrow-17.0.0-9.el10_2.x86_64 2026-02-22T20:32:10.617 INFO:teuthology.orchestra.run.trial084.stdout: libarrow-doc-17.0.0-9.el10_2.noarch 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: libcephfs2-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: libnbd-1.22.2-3.el10_1.x86_64 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: liboath-2.6.12-1.el10_0.x86_64 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: liborc2-2.0.7-1.el10_2.x86_64 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: libpmem-2.1.0-2.el10_0.x86_64 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: libpmemobj-2.1.0-2.el10_0.x86_64 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: librabbitmq-0.14.0-3.el10.x86_64 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: libradosstriper1-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: librdkafka-2.1.1-7.el10.x86_64 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: librgw2-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: libunwind-1.8.0-4.el10_0.x86_64 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: lttng-ust-2.13.7-5.el10.x86_64 2026-02-22T20:32:10.618 INFO:teuthology.orchestra.run.trial084.stdout: mailcap-2.1.54-8.el10.noarch 2026-02-22T20:32:10.619 INFO:teuthology.orchestra.run.trial084.stdout: ndctl-libs-80-4.el10.x86_64 2026-02-22T20:32:10.619 INFO:teuthology.orchestra.run.trial084.stdout: parquet-libs-17.0.0-9.el10_2.x86_64 2026-02-22T20:32:10.619 INFO:teuthology.orchestra.run.trial084.stdout: pciutils-3.13.0-6.el10.x86_64 2026-02-22T20:32:10.619 INFO:teuthology.orchestra.run.trial084.stdout: protobuf-3.19.6-11.el10.x86_64 2026-02-22T20:32:10.619 INFO:teuthology.orchestra.run.trial084.stdout: python3-ceph-argparse-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.619 INFO:teuthology.orchestra.run.trial084.stdout: python3-ceph-common-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.619 INFO:teuthology.orchestra.run.trial084.stdout: python3-cephfs-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.619 INFO:teuthology.orchestra.run.trial084.stdout: python3-prettytable-3.6.0-10.el10.noarch 2026-02-22T20:32:10.619 INFO:teuthology.orchestra.run.trial084.stdout: python3-rados-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.619 INFO:teuthology.orchestra.run.trial084.stdout: python3-rbd-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.619 INFO:teuthology.orchestra.run.trial084.stdout: python3-rgw-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:10.619 INFO:teuthology.orchestra.run.trial084.stdout: python3-wcwidth-0.2.6-6.el10.noarch 2026-02-22T20:32:10.620 INFO:teuthology.orchestra.run.trial084.stdout: qatlib-24.09.0-2.el10.x86_64 2026-02-22T20:32:10.620 INFO:teuthology.orchestra.run.trial084.stdout: qatlib-service-24.09.0-2.el10.x86_64 2026-02-22T20:32:10.620 INFO:teuthology.orchestra.run.trial084.stdout: qatzip-libs-1.3.1-1.el10.x86_64 2026-02-22T20:32:10.620 INFO:teuthology.orchestra.run.trial084.stdout: re2-2:20251105-1.el10_2.x86_64 2026-02-22T20:32:10.620 INFO:teuthology.orchestra.run.trial084.stdout: thrift-0.20.0-4.el10_1.x86_64 2026-02-22T20:32:10.620 INFO:teuthology.orchestra.run.trial084.stdout: utf8proc-2.7.0-9.el10.x86_64 2026-02-22T20:32:10.620 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:10.620 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:10.657 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install ceph-test 2026-02-22T20:32:10.946 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:22 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:10.984 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:10.984 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:10.985 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repository Size 2026-02-22T20:32:10.985 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:10.985 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:10.985 INFO:teuthology.orchestra.run.trial084.stdout: ceph-test x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 86 M 2026-02-22T20:32:10.985 INFO:teuthology.orchestra.run.trial084.stdout:Installing dependencies: 2026-02-22T20:32:10.985 INFO:teuthology.orchestra.run.trial084.stdout: libcephsqlite x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 172 k 2026-02-22T20:32:10.985 INFO:teuthology.orchestra.run.trial084.stdout: libxslt x86_64 1.1.39-8.el10_0 appstream 189 k 2026-02-22T20:32:10.985 INFO:teuthology.orchestra.run.trial084.stdout: socat x86_64 1.7.4.4-8.el10 appstream 303 k 2026-02-22T20:32:10.985 INFO:teuthology.orchestra.run.trial084.stdout: xmlstarlet x86_64 1.6.1-27.el10 appstream 63 k 2026-02-22T20:32:10.985 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:10.985 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:10.986 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:10.986 INFO:teuthology.orchestra.run.trial084.stdout:Install 5 Packages 2026-02-22T20:32:10.986 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:10.986 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 87 M 2026-02-22T20:32:10.986 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 349 M 2026-02-22T20:32:10.986 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:11.022 INFO:teuthology.orchestra.run.trial084.stdout:(1/5): libxslt-1.1.39-8.el10_0.x86_64.rpm 5.0 MB/s | 189 kB 00:00 2026-02-22T20:32:11.043 INFO:teuthology.orchestra.run.trial084.stdout:(2/5): socat-1.7.4.4-8.el10.x86_64.rpm 14 MB/s | 303 kB 00:00 2026-02-22T20:32:11.058 INFO:teuthology.orchestra.run.trial084.stdout:(3/5): xmlstarlet-1.6.1-27.el10.x86_64.rpm 4.1 MB/s | 63 kB 00:00 2026-02-22T20:32:11.181 INFO:teuthology.orchestra.run.trial084.stdout:(4/5): libcephsqlite-20.2.0-794.ga5c3084f.el10. 880 kB/s | 172 kB 00:00 2026-02-22T20:32:22.074 INFO:teuthology.orchestra.run.trial084.stdout:(5/5): ceph-test-20.2.0-794.ga5c3084f.el10.x86_ 7.8 MB/s | 86 MB 00:11 2026-02-22T20:32:22.075 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:22.075 INFO:teuthology.orchestra.run.trial084.stdout:Total 7.8 MB/s | 87 MB 00:11 2026-02-22T20:32:22.075 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:22.085 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:22.085 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:22.372 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:22.372 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:22.683 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:22.687 INFO:teuthology.orchestra.run.trial084.stdout: Installing : socat-1.7.4.4-8.el10.x86_64 1/5 2026-02-22T20:32:22.689 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libxslt-1.1.39-8.el10_0.x86_64 2/5 2026-02-22T20:32:22.695 INFO:teuthology.orchestra.run.trial084.stdout: Installing : xmlstarlet-1.6.1-27.el10.x86_64 3/5 2026-02-22T20:32:22.696 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libcephsqlite-2:20.2.0-794.ga5c3084f.el10.x86_64 4/5 2026-02-22T20:32:25.460 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: libcephsqlite-2:20.2.0-794.ga5c3084f.el10.x86_64 4/5 2026-02-22T20:32:25.479 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-test-2:20.2.0-794.ga5c3084f.el10.x86_64 5/5 2026-02-22T20:32:25.830 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-test-2:20.2.0-794.ga5c3084f.el10.x86_64 5/5 2026-02-22T20:32:25.830 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:25.830 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:25.831 INFO:teuthology.orchestra.run.trial084.stdout: ceph-test-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:25.831 INFO:teuthology.orchestra.run.trial084.stdout: libcephsqlite-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:25.831 INFO:teuthology.orchestra.run.trial084.stdout: libxslt-1.1.39-8.el10_0.x86_64 2026-02-22T20:32:25.831 INFO:teuthology.orchestra.run.trial084.stdout: socat-1.7.4.4-8.el10.x86_64 2026-02-22T20:32:25.831 INFO:teuthology.orchestra.run.trial084.stdout: xmlstarlet-1.6.1-27.el10.x86_64 2026-02-22T20:32:25.831 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:25.831 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:25.869 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install ceph 2026-02-22T20:32:26.180 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:38 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:26.219 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:26.220 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:26.221 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repository Size 2026-02-22T20:32:26.221 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:26.221 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:26.221 INFO:teuthology.orchestra.run.trial084.stdout: ceph x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 6.6 k 2026-02-22T20:32:26.221 INFO:teuthology.orchestra.run.trial084.stdout:Installing dependencies: 2026-02-22T20:32:26.221 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mds x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 2.4 M 2026-02-22T20:32:26.221 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mgr x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 997 k 2026-02-22T20:32:26.221 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mgr-modules-core noarch 2:20.2.0-794.ga5c3084f.el10 ceph-noarch 295 k 2026-02-22T20:32:26.221 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mon x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 5.2 M 2026-02-22T20:32:26.222 INFO:teuthology.orchestra.run.trial084.stdout: ceph-osd x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 18 M 2026-02-22T20:32:26.222 INFO:teuthology.orchestra.run.trial084.stdout: ledmon-libs x86_64 1.1.0-2.el10 appstream 42 k 2026-02-22T20:32:26.222 INFO:teuthology.orchestra.run.trial084.stdout: libconfig x86_64 1.7.3-10.el10 baseos 72 k 2026-02-22T20:32:26.222 INFO:teuthology.orchestra.run.trial084.stdout: libstoragemgmt x86_64 1.10.1-4.el10 appstream 251 k 2026-02-22T20:32:26.222 INFO:teuthology.orchestra.run.trial084.stdout: lua x86_64 5.4.6-7.el10 appstream 190 k 2026-02-22T20:32:26.222 INFO:teuthology.orchestra.run.trial084.stdout: lua-devel x86_64 5.4.6-7.el10 crb 22 k 2026-02-22T20:32:26.222 INFO:teuthology.orchestra.run.trial084.stdout: luarocks noarch 3.9.2-6.el10_0 epel 152 k 2026-02-22T20:32:26.222 INFO:teuthology.orchestra.run.trial084.stdout: python3-autocommand noarch 2.2.2-9.el10_0 epel 32 k 2026-02-22T20:32:26.222 INFO:teuthology.orchestra.run.trial084.stdout: python3-bcrypt x86_64 4.3.0-3.el10_1 epel 238 k 2026-02-22T20:32:26.222 INFO:teuthology.orchestra.run.trial084.stdout: python3-cffi x86_64 1.16.0-7.el10 baseos 310 k 2026-02-22T20:32:26.223 INFO:teuthology.orchestra.run.trial084.stdout: python3-cheroot noarch 10.0.1-4.el10_0 epel 215 k 2026-02-22T20:32:26.223 INFO:teuthology.orchestra.run.trial084.stdout: python3-cherrypy noarch 18.10.0-4.el10_0 epel 359 k 2026-02-22T20:32:26.223 INFO:teuthology.orchestra.run.trial084.stdout: python3-cryptography x86_64 43.0.0-4.el10 baseos 1.3 M 2026-02-22T20:32:26.223 INFO:teuthology.orchestra.run.trial084.stdout: python3-jaraco-collections 2026-02-22T20:32:26.223 INFO:teuthology.orchestra.run.trial084.stdout: noarch 5.1.0-1.el10_0 epel 29 k 2026-02-22T20:32:26.223 INFO:teuthology.orchestra.run.trial084.stdout: python3-jaraco-context noarch 6.0.1-4.el10_0 epel 21 k 2026-02-22T20:32:26.223 INFO:teuthology.orchestra.run.trial084.stdout: python3-jaraco-functools noarch 4.0.2-4.el10_0 epel 23 k 2026-02-22T20:32:26.223 INFO:teuthology.orchestra.run.trial084.stdout: python3-jaraco-text noarch 4.0.0-3.el10_0 epel 29 k 2026-02-22T20:32:26.223 INFO:teuthology.orchestra.run.trial084.stdout: python3-libstoragemgmt x86_64 1.10.1-4.el10 appstream 219 k 2026-02-22T20:32:26.223 INFO:teuthology.orchestra.run.trial084.stdout: python3-more-itertools noarch 10.1.0-8.el10_0 epel 100 k 2026-02-22T20:32:26.223 INFO:teuthology.orchestra.run.trial084.stdout: python3-packaging noarch 24.2-2.el10 baseos 155 k 2026-02-22T20:32:26.224 INFO:teuthology.orchestra.run.trial084.stdout: python3-ply noarch 3.11-25.el10 baseos 137 k 2026-02-22T20:32:26.224 INFO:teuthology.orchestra.run.trial084.stdout: python3-portend noarch 3.2.0-6.el10_0 epel 18 k 2026-02-22T20:32:26.224 INFO:teuthology.orchestra.run.trial084.stdout: python3-pyOpenSSL noarch 25.0.0-1.el10_1 epel 113 k 2026-02-22T20:32:26.224 INFO:teuthology.orchestra.run.trial084.stdout: python3-pycparser noarch 2.20-16.el10 baseos 160 k 2026-02-22T20:32:26.224 INFO:teuthology.orchestra.run.trial084.stdout: python3-requests noarch 2.32.4-1.el10_0 baseos 160 k 2026-02-22T20:32:26.224 INFO:teuthology.orchestra.run.trial084.stdout: python3-tempora noarch 5.7.0-1.el10_0 epel 37 k 2026-02-22T20:32:26.224 INFO:teuthology.orchestra.run.trial084.stdout: python3-typing-extensions noarch 4.9.0-6.el10 baseos 77 k 2026-02-22T20:32:26.224 INFO:teuthology.orchestra.run.trial084.stdout: python3-urllib3 noarch 1.26.19-2.el10_1.1 baseos 285 k 2026-02-22T20:32:26.224 INFO:teuthology.orchestra.run.trial084.stdout: python3-zc-lockfile noarch 3.0.post1-11.el10_0 epel 23 k 2026-02-22T20:32:26.224 INFO:teuthology.orchestra.run.trial084.stdout: unzip x86_64 6.0-69.el10 baseos 188 k 2026-02-22T20:32:26.224 INFO:teuthology.orchestra.run.trial084.stdout: zip x86_64 3.0-45.el10 baseos 271 k 2026-02-22T20:32:26.225 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:26.225 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:26.225 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:26.225 INFO:teuthology.orchestra.run.trial084.stdout:Install 36 Packages 2026-02-22T20:32:26.225 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:26.226 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 32 M 2026-02-22T20:32:26.226 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 103 M 2026-02-22T20:32:26.226 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:26.751 INFO:teuthology.orchestra.run.trial084.stdout:(1/36): ceph-20.2.0-794.ga5c3084f.el10.x86_64.r 36 kB/s | 6.6 kB 00:00 2026-02-22T20:32:27.110 INFO:teuthology.orchestra.run.trial084.stdout:(2/36): ceph-mgr-20.2.0-794.ga5c3084f.el10.x86_ 1.8 MB/s | 997 kB 00:00 2026-02-22T20:32:27.326 INFO:teuthology.orchestra.run.trial084.stdout:(3/36): ceph-mds-20.2.0-794.ga5c3084f.el10.x86_ 3.1 MB/s | 2.4 MB 00:00 2026-02-22T20:32:27.401 INFO:teuthology.orchestra.run.trial084.stdout:(4/36): ceph-mgr-modules-core-20.2.0-794.ga5c30 3.9 MB/s | 295 kB 00:00 2026-02-22T20:32:27.434 INFO:teuthology.orchestra.run.trial084.stdout:(5/36): ledmon-libs-1.1.0-2.el10.x86_64.rpm 1.2 MB/s | 42 kB 00:00 2026-02-22T20:32:27.455 INFO:teuthology.orchestra.run.trial084.stdout:(6/36): libstoragemgmt-1.10.1-4.el10.x86_64.rpm 12 MB/s | 251 kB 00:00 2026-02-22T20:32:27.473 INFO:teuthology.orchestra.run.trial084.stdout:(7/36): lua-5.4.6-7.el10.x86_64.rpm 10 MB/s | 190 kB 00:00 2026-02-22T20:32:27.492 INFO:teuthology.orchestra.run.trial084.stdout:(8/36): python3-libstoragemgmt-1.10.1-4.el10.x8 12 MB/s | 219 kB 00:00 2026-02-22T20:32:27.508 INFO:teuthology.orchestra.run.trial084.stdout:(9/36): libconfig-1.7.3-10.el10.x86_64.rpm 4.3 MB/s | 72 kB 00:00 2026-02-22T20:32:27.526 INFO:teuthology.orchestra.run.trial084.stdout:(10/36): python3-cffi-1.16.0-7.el10.x86_64.rpm 17 MB/s | 310 kB 00:00 2026-02-22T20:32:27.563 INFO:teuthology.orchestra.run.trial084.stdout:(11/36): python3-cryptography-43.0.0-4.el10.x86 36 MB/s | 1.3 MB 00:00 2026-02-22T20:32:27.580 INFO:teuthology.orchestra.run.trial084.stdout:(12/36): python3-packaging-24.2-2.el10.noarch.r 9.3 MB/s | 155 kB 00:00 2026-02-22T20:32:27.596 INFO:teuthology.orchestra.run.trial084.stdout:(13/36): python3-ply-3.11-25.el10.noarch.rpm 8.2 MB/s | 137 kB 00:00 2026-02-22T20:32:27.626 INFO:teuthology.orchestra.run.trial084.stdout:(14/36): python3-pycparser-2.20-16.el10.noarch. 5.2 MB/s | 160 kB 00:00 2026-02-22T20:32:27.632 INFO:teuthology.orchestra.run.trial084.stdout:(15/36): ceph-mon-20.2.0-794.ga5c3084f.el10.x86 5.9 MB/s | 5.2 MB 00:00 2026-02-22T20:32:27.644 INFO:teuthology.orchestra.run.trial084.stdout:(16/36): python3-requests-2.32.4-1.el10_0.noarc 8.9 MB/s | 160 kB 00:00 2026-02-22T20:32:27.649 INFO:teuthology.orchestra.run.trial084.stdout:(17/36): python3-typing-extensions-4.9.0-6.el10 4.5 MB/s | 77 kB 00:00 2026-02-22T20:32:27.663 INFO:teuthology.orchestra.run.trial084.stdout:(18/36): python3-urllib3-1.26.19-2.el10_1.1.noa 15 MB/s | 285 kB 00:00 2026-02-22T20:32:27.665 INFO:teuthology.orchestra.run.trial084.stdout:(19/36): unzip-6.0-69.el10.x86_64.rpm 11 MB/s | 188 kB 00:00 2026-02-22T20:32:27.678 INFO:teuthology.orchestra.run.trial084.stdout:(20/36): lua-devel-5.4.6-7.el10.x86_64.rpm 1.8 MB/s | 22 kB 00:00 2026-02-22T20:32:27.680 INFO:teuthology.orchestra.run.trial084.stdout:(21/36): zip-3.0-45.el10.x86_64.rpm 16 MB/s | 271 kB 00:00 2026-02-22T20:32:27.805 INFO:teuthology.orchestra.run.trial084.stdout:(22/36): python3-autocommand-2.2.2-9.el10_0.noa 254 kB/s | 32 kB 00:00 2026-02-22T20:32:27.883 INFO:teuthology.orchestra.run.trial084.stdout:(23/36): luarocks-3.9.2-6.el10_0.noarch.rpm 741 kB/s | 152 kB 00:00 2026-02-22T20:32:27.950 INFO:teuthology.orchestra.run.trial084.stdout:(24/36): python3-bcrypt-4.3.0-3.el10_1.x86_64.r 1.6 MB/s | 238 kB 00:00 2026-02-22T20:32:27.966 INFO:teuthology.orchestra.run.trial084.stdout:(25/36): python3-cheroot-10.0.1-4.el10_0.noarch 2.5 MB/s | 215 kB 00:00 2026-02-22T20:32:28.003 INFO:teuthology.orchestra.run.trial084.stdout:(26/36): python3-jaraco-collections-5.1.0-1.el1 784 kB/s | 29 kB 00:00 2026-02-22T20:32:28.035 INFO:teuthology.orchestra.run.trial084.stdout:(27/36): python3-cherrypy-18.10.0-4.el10_0.noar 4.1 MB/s | 359 kB 00:00 2026-02-22T20:32:28.053 INFO:teuthology.orchestra.run.trial084.stdout:(28/36): python3-jaraco-context-6.0.1-4.el10_0. 426 kB/s | 21 kB 00:00 2026-02-22T20:32:28.091 INFO:teuthology.orchestra.run.trial084.stdout:(29/36): python3-jaraco-functools-4.0.2-4.el10_ 403 kB/s | 23 kB 00:00 2026-02-22T20:32:28.092 INFO:teuthology.orchestra.run.trial084.stdout:(30/36): python3-jaraco-text-4.0.0-3.el10_0.noa 728 kB/s | 29 kB 00:00 2026-02-22T20:32:28.132 INFO:teuthology.orchestra.run.trial084.stdout:(31/36): python3-portend-3.2.0-6.el10_0.noarch. 445 kB/s | 18 kB 00:00 2026-02-22T20:32:28.136 INFO:teuthology.orchestra.run.trial084.stdout:(32/36): python3-more-itertools-10.1.0-8.el10_0 2.2 MB/s | 100 kB 00:00 2026-02-22T20:32:28.209 INFO:teuthology.orchestra.run.trial084.stdout:(33/36): python3-tempora-5.7.0-1.el10_0.noarch. 511 kB/s | 37 kB 00:00 2026-02-22T20:32:28.214 INFO:teuthology.orchestra.run.trial084.stdout:(34/36): python3-pyOpenSSL-25.0.0-1.el10_1.noar 1.4 MB/s | 113 kB 00:00 2026-02-22T20:32:28.264 INFO:teuthology.orchestra.run.trial084.stdout:(35/36): ceph-osd-20.2.0-794.ga5c3084f.el10.x86 15 MB/s | 18 MB 00:01 2026-02-22T20:32:28.265 INFO:teuthology.orchestra.run.trial084.stdout:(36/36): python3-zc-lockfile-3.0.post1-11.el10_ 414 kB/s | 23 kB 00:00 2026-02-22T20:32:28.265 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:28.266 INFO:teuthology.orchestra.run.trial084.stdout:Total 15 MB/s | 32 MB 00:02 2026-02-22T20:32:28.266 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:28.284 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:28.284 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:28.429 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:28.429 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:28.606 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:28.608 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-more-itertools-10.1.0-8.el10_0.noarch 1/36 2026-02-22T20:32:28.612 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-jaraco-functools-4.0.2-4.el10_0.noarch 2/36 2026-02-22T20:32:28.615 INFO:teuthology.orchestra.run.trial084.stdout: Installing : unzip-6.0-69.el10.x86_64 3/36 2026-02-22T20:32:28.617 INFO:teuthology.orchestra.run.trial084.stdout: Installing : lua-5.4.6-7.el10.x86_64 4/36 2026-02-22T20:32:28.619 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ledmon-libs-1.1.0-2.el10.x86_64 5/36 2026-02-22T20:32:28.622 INFO:teuthology.orchestra.run.trial084.stdout: Installing : lua-devel-5.4.6-7.el10.x86_64 6/36 2026-02-22T20:32:28.631 INFO:teuthology.orchestra.run.trial084.stdout: Installing : zip-3.0-45.el10.x86_64 7/36 2026-02-22T20:32:28.635 INFO:teuthology.orchestra.run.trial084.stdout: Installing : luarocks-3.9.2-6.el10_0.noarch 8/36 2026-02-22T20:32:28.637 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-tempora-5.7.0-1.el10_0.noarch 9/36 2026-02-22T20:32:28.639 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-portend-3.2.0-6.el10_0.noarch 10/36 2026-02-22T20:32:28.641 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-zc-lockfile-3.0.post1-11.el10_0.noarch 11/36 2026-02-22T20:32:28.644 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-jaraco-context-6.0.1-4.el10_0.noarch 12/36 2026-02-22T20:32:28.647 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-bcrypt-4.3.0-3.el10_1.x86_64 13/36 2026-02-22T20:32:28.650 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-autocommand-2.2.2-9.el10_0.noarch 14/36 2026-02-22T20:32:28.652 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-jaraco-text-4.0.0-3.el10_0.noarch 15/36 2026-02-22T20:32:28.661 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-jaraco-collections-5.1.0-1.el10_0.noarch 16/36 2026-02-22T20:32:28.666 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-urllib3-1.26.19-2.el10_1.1.noarch 17/36 2026-02-22T20:32:28.669 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-requests-2.32.4-1.el10_0.noarch 18/36 2026-02-22T20:32:28.672 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-typing-extensions-4.9.0-6.el10.noarch 19/36 2026-02-22T20:32:28.678 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-ply-3.11-25.el10.noarch 20/36 2026-02-22T20:32:28.685 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-pycparser-2.20-16.el10.noarch 21/36 2026-02-22T20:32:28.716 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-cffi-1.16.0-7.el10.x86_64 22/36 2026-02-22T20:32:28.722 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-cryptography-43.0.0-4.el10.x86_64 23/36 2026-02-22T20:32:28.730 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-pyOpenSSL-25.0.0-1.el10_1.noarch 24/36 2026-02-22T20:32:28.741 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-cheroot-10.0.1-4.el10_0.noarch 25/36 2026-02-22T20:32:28.747 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-cherrypy-18.10.0-4.el10_0.noarch 26/36 2026-02-22T20:32:28.770 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-packaging-24.2-2.el10.noarch 27/36 2026-02-22T20:32:28.803 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-mgr-modules-core-2:20.2.0-794.ga5c3084f.el10. 28/36 2026-02-22T20:32:28.804 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-mgr-2:20.2.0-794.ga5c3084f.el10.x86_64 29/36 2026-02-22T20:32:28.828 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-mgr-2:20.2.0-794.ga5c3084f.el10.x86_64 29/36 2026-02-22T20:32:28.828 INFO:teuthology.orchestra.run.trial084.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-22T20:32:28.828 INFO:teuthology.orchestra.run.trial084.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-22T20:32:28.828 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph-mgr.target' → '/usr/lib/systemd/system/ceph-mgr.target'. 2026-02-22T20:32:28.828 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-mgr.target' → '/usr/lib/systemd/system/ceph-mgr.target'. 2026-02-22T20:32:28.828 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:28.832 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libconfig-1.7.3-10.el10.x86_64 30/36 2026-02-22T20:32:28.852 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: libstoragemgmt-1.10.1-4.el10.x86_64 31/36 2026-02-22T20:32:28.852 INFO:teuthology.orchestra.run.trial084.stdout:Creating group 'libstoragemgmt' with GID 992. 2026-02-22T20:32:28.852 INFO:teuthology.orchestra.run.trial084.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 992 and GID 992. 2026-02-22T20:32:28.852 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:28.857 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libstoragemgmt-1.10.1-4.el10.x86_64 31/36 2026-02-22T20:32:28.879 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: libstoragemgmt-1.10.1-4.el10.x86_64 31/36 2026-02-22T20:32:28.879 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/libstoragemgmt.service' → '/usr/lib/systemd/system/libstoragemgmt.service'. 2026-02-22T20:32:28.879 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:29.385 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-libstoragemgmt-1.10.1-4.el10.x86_64 32/36 2026-02-22T20:32:29.386 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-osd-2:20.2.0-794.ga5c3084f.el10.x86_64 33/36 2026-02-22T20:32:29.411 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-osd-2:20.2.0-794.ga5c3084f.el10.x86_64 33/36 2026-02-22T20:32:29.411 INFO:teuthology.orchestra.run.trial084.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-22T20:32:29.411 INFO:teuthology.orchestra.run.trial084.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-22T20:32:29.411 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph-osd.target' → '/usr/lib/systemd/system/ceph-osd.target'. 2026-02-22T20:32:29.411 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-osd.target' → '/usr/lib/systemd/system/ceph-osd.target'. 2026-02-22T20:32:29.411 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:29.571 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-mon-2:20.2.0-794.ga5c3084f.el10.x86_64 34/36 2026-02-22T20:32:29.594 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-mon-2:20.2.0-794.ga5c3084f.el10.x86_64 34/36 2026-02-22T20:32:29.594 INFO:teuthology.orchestra.run.trial084.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-22T20:32:29.594 INFO:teuthology.orchestra.run.trial084.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-22T20:32:29.594 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph-mon.target' → '/usr/lib/systemd/system/ceph-mon.target'. 2026-02-22T20:32:29.594 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-mon.target' → '/usr/lib/systemd/system/ceph-mon.target'. 2026-02-22T20:32:29.594 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:29.666 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-mds-2:20.2.0-794.ga5c3084f.el10.x86_64 35/36 2026-02-22T20:32:29.688 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-mds-2:20.2.0-794.ga5c3084f.el10.x86_64 35/36 2026-02-22T20:32:29.688 INFO:teuthology.orchestra.run.trial084.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-22T20:32:29.688 INFO:teuthology.orchestra.run.trial084.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-22T20:32:29.688 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph-mds.target' → '/usr/lib/systemd/system/ceph-mds.target'. 2026-02-22T20:32:29.688 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-mds.target' → '/usr/lib/systemd/system/ceph-mds.target'. 2026-02-22T20:32:29.688 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:29.709 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-2:20.2.0-794.ga5c3084f.el10.x86_64 36/36 2026-02-22T20:32:30.178 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-2:20.2.0-794.ga5c3084f.el10.x86_64 36/36 2026-02-22T20:32:30.178 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:30.178 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:30.178 INFO:teuthology.orchestra.run.trial084.stdout: ceph-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:30.178 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mds-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:30.178 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mgr-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:30.179 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mgr-modules-core-2:20.2.0-794.ga5c3084f.el10.noarch 2026-02-22T20:32:30.179 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mon-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:30.179 INFO:teuthology.orchestra.run.trial084.stdout: ceph-osd-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:30.179 INFO:teuthology.orchestra.run.trial084.stdout: ledmon-libs-1.1.0-2.el10.x86_64 2026-02-22T20:32:30.179 INFO:teuthology.orchestra.run.trial084.stdout: libconfig-1.7.3-10.el10.x86_64 2026-02-22T20:32:30.179 INFO:teuthology.orchestra.run.trial084.stdout: libstoragemgmt-1.10.1-4.el10.x86_64 2026-02-22T20:32:30.179 INFO:teuthology.orchestra.run.trial084.stdout: lua-5.4.6-7.el10.x86_64 2026-02-22T20:32:30.179 INFO:teuthology.orchestra.run.trial084.stdout: lua-devel-5.4.6-7.el10.x86_64 2026-02-22T20:32:30.179 INFO:teuthology.orchestra.run.trial084.stdout: luarocks-3.9.2-6.el10_0.noarch 2026-02-22T20:32:30.179 INFO:teuthology.orchestra.run.trial084.stdout: python3-autocommand-2.2.2-9.el10_0.noarch 2026-02-22T20:32:30.179 INFO:teuthology.orchestra.run.trial084.stdout: python3-bcrypt-4.3.0-3.el10_1.x86_64 2026-02-22T20:32:30.179 INFO:teuthology.orchestra.run.trial084.stdout: python3-cffi-1.16.0-7.el10.x86_64 2026-02-22T20:32:30.180 INFO:teuthology.orchestra.run.trial084.stdout: python3-cheroot-10.0.1-4.el10_0.noarch 2026-02-22T20:32:30.180 INFO:teuthology.orchestra.run.trial084.stdout: python3-cherrypy-18.10.0-4.el10_0.noarch 2026-02-22T20:32:30.180 INFO:teuthology.orchestra.run.trial084.stdout: python3-cryptography-43.0.0-4.el10.x86_64 2026-02-22T20:32:30.180 INFO:teuthology.orchestra.run.trial084.stdout: python3-jaraco-collections-5.1.0-1.el10_0.noarch 2026-02-22T20:32:30.180 INFO:teuthology.orchestra.run.trial084.stdout: python3-jaraco-context-6.0.1-4.el10_0.noarch 2026-02-22T20:32:30.180 INFO:teuthology.orchestra.run.trial084.stdout: python3-jaraco-functools-4.0.2-4.el10_0.noarch 2026-02-22T20:32:30.180 INFO:teuthology.orchestra.run.trial084.stdout: python3-jaraco-text-4.0.0-3.el10_0.noarch 2026-02-22T20:32:30.180 INFO:teuthology.orchestra.run.trial084.stdout: python3-libstoragemgmt-1.10.1-4.el10.x86_64 2026-02-22T20:32:30.180 INFO:teuthology.orchestra.run.trial084.stdout: python3-more-itertools-10.1.0-8.el10_0.noarch 2026-02-22T20:32:30.180 INFO:teuthology.orchestra.run.trial084.stdout: python3-packaging-24.2-2.el10.noarch 2026-02-22T20:32:30.180 INFO:teuthology.orchestra.run.trial084.stdout: python3-ply-3.11-25.el10.noarch 2026-02-22T20:32:30.180 INFO:teuthology.orchestra.run.trial084.stdout: python3-portend-3.2.0-6.el10_0.noarch 2026-02-22T20:32:30.181 INFO:teuthology.orchestra.run.trial084.stdout: python3-pyOpenSSL-25.0.0-1.el10_1.noarch 2026-02-22T20:32:30.181 INFO:teuthology.orchestra.run.trial084.stdout: python3-pycparser-2.20-16.el10.noarch 2026-02-22T20:32:30.181 INFO:teuthology.orchestra.run.trial084.stdout: python3-requests-2.32.4-1.el10_0.noarch 2026-02-22T20:32:30.181 INFO:teuthology.orchestra.run.trial084.stdout: python3-tempora-5.7.0-1.el10_0.noarch 2026-02-22T20:32:30.181 INFO:teuthology.orchestra.run.trial084.stdout: python3-typing-extensions-4.9.0-6.el10.noarch 2026-02-22T20:32:30.181 INFO:teuthology.orchestra.run.trial084.stdout: python3-urllib3-1.26.19-2.el10_1.1.noarch 2026-02-22T20:32:30.181 INFO:teuthology.orchestra.run.trial084.stdout: python3-zc-lockfile-3.0.post1-11.el10_0.noarch 2026-02-22T20:32:30.181 INFO:teuthology.orchestra.run.trial084.stdout: unzip-6.0-69.el10.x86_64 2026-02-22T20:32:30.181 INFO:teuthology.orchestra.run.trial084.stdout: zip-3.0-45.el10.x86_64 2026-02-22T20:32:30.181 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:30.181 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:30.212 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install ceph-base 2026-02-22T20:32:30.503 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:42 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:30.531 INFO:teuthology.orchestra.run.trial084.stdout:Package ceph-base-2:20.2.0-794.ga5c3084f.el10.x86_64 is already installed. 2026-02-22T20:32:30.540 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:30.540 INFO:teuthology.orchestra.run.trial084.stdout:Nothing to do. 2026-02-22T20:32:30.540 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:30.560 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install cephadm 2026-02-22T20:32:30.841 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:42 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:30.878 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:30.878 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:30.878 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repository Size 2026-02-22T20:32:30.878 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:30.878 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:30.878 INFO:teuthology.orchestra.run.trial084.stdout: cephadm noarch 2:20.2.0-794.ga5c3084f.el10 ceph-noarch 1.3 M 2026-02-22T20:32:30.879 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:30.879 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:30.879 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:30.879 INFO:teuthology.orchestra.run.trial084.stdout:Install 1 Package 2026-02-22T20:32:30.879 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:30.879 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 1.3 M 2026-02-22T20:32:30.879 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 1.3 M 2026-02-22T20:32:30.879 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:31.287 INFO:teuthology.orchestra.run.trial084.stdout:cephadm-20.2.0-794.ga5c3084f.el10.noarch.rpm 3.2 MB/s | 1.3 MB 00:00 2026-02-22T20:32:31.287 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:31.287 INFO:teuthology.orchestra.run.trial084.stdout:Total 3.2 MB/s | 1.3 MB 00:00 2026-02-22T20:32:31.288 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:31.291 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:31.291 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:31.296 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:31.296 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:31.317 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:31.353 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: cephadm-2:20.2.0-794.ga5c3084f.el10.noarch 1/1 2026-02-22T20:32:31.369 INFO:teuthology.orchestra.run.trial084.stdout: Installing : cephadm-2:20.2.0-794.ga5c3084f.el10.noarch 1/1 2026-02-22T20:32:31.465 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: cephadm-2:20.2.0-794.ga5c3084f.el10.noarch 1/1 2026-02-22T20:32:31.466 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:31.466 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:31.466 INFO:teuthology.orchestra.run.trial084.stdout: cephadm-2:20.2.0-794.ga5c3084f.el10.noarch 2026-02-22T20:32:31.466 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:31.466 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:31.489 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install ceph-immutable-object-cache 2026-02-22T20:32:31.781 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:43 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:31.816 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:31.816 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:31.816 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repo Size 2026-02-22T20:32:31.816 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:31.816 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:31.817 INFO:teuthology.orchestra.run.trial084.stdout: ceph-immutable-object-cache x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 155 k 2026-02-22T20:32:31.817 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:31.817 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:31.817 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:31.817 INFO:teuthology.orchestra.run.trial084.stdout:Install 1 Package 2026-02-22T20:32:31.817 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:31.817 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 155 k 2026-02-22T20:32:31.817 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 441 k 2026-02-22T20:32:31.817 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:32.098 INFO:teuthology.orchestra.run.trial084.stdout:ceph-immutable-object-cache-20.2.0-794.ga5c3084 554 kB/s | 155 kB 00:00 2026-02-22T20:32:32.098 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:32.098 INFO:teuthology.orchestra.run.trial084.stdout:Total 552 kB/s | 155 kB 00:00 2026-02-22T20:32:32.098 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:32.103 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:32.103 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:32.127 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:32.127 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:32.174 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:32.175 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-immutable-object-cache-2:20.2.0-794.ga5c3084f.e 1/1 2026-02-22T20:32:32.198 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.2.0-794.ga5c3084f.e 1/1 2026-02-22T20:32:32.199 INFO:teuthology.orchestra.run.trial084.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-22T20:32:32.199 INFO:teuthology.orchestra.run.trial084.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-22T20:32:32.199 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:32.467 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:32.467 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:32.467 INFO:teuthology.orchestra.run.trial084.stdout: ceph-immutable-object-cache-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:32.468 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:32.468 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:32.494 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install ceph-mgr 2026-02-22T20:32:32.781 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:44 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:32.807 INFO:teuthology.orchestra.run.trial084.stdout:Package ceph-mgr-2:20.2.0-794.ga5c3084f.el10.x86_64 is already installed. 2026-02-22T20:32:32.816 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:32.816 INFO:teuthology.orchestra.run.trial084.stdout:Nothing to do. 2026-02-22T20:32:32.817 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:32.836 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install ceph-mgr-dashboard 2026-02-22T20:32:33.123 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:45 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:33.157 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repository Size 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mgr-dashboard noarch 2:20.2.0-794.ga5c3084f.el10 ceph-noarch 11 M 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout:Installing dependencies: 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: ceph-grafana-dashboards noarch 2:20.2.0-794.ga5c3084f.el10 ceph-noarch 43 k 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: ceph-prometheus-alerts noarch 2:20.2.0-794.ga5c3084f.el10 ceph-noarch 17 k 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: grpc-data noarch 1.48.4-44.el10_0 epel 21 k 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: protobuf-compiler x86_64 3.19.6-11.el10 crb 787 k 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: python3-grpcio x86_64 1.48.4-44.el10_0 epel 2.3 M 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: python3-grpcio-tools x86_64 1.48.4-44.el10_0 epel 157 k 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: python3-jmespath noarch 1.0.1-8.el10 appstream 60 k 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: python3-protobuf x86_64 3.19.6-11.el10 appstream 419 k 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: python3-repoze-lru noarch 0.7-23.el10_0 epel 36 k 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: python3-routes noarch 2.5.1-11.el10_0 epel 201 k 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: python3-xmltodict noarch 0.13.0-5.el10_0 epel 26 k 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:33.158 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:33.159 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:33.159 INFO:teuthology.orchestra.run.trial084.stdout:Install 12 Packages 2026-02-22T20:32:33.159 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:33.159 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 15 M 2026-02-22T20:32:33.159 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 154 M 2026-02-22T20:32:33.159 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:33.414 INFO:teuthology.orchestra.run.trial084.stdout:(1/12): ceph-prometheus-alerts-20.2.0-794.ga5c3 97 kB/s | 17 kB 00:00 2026-02-22T20:32:33.443 INFO:teuthology.orchestra.run.trial084.stdout:(2/12): ceph-grafana-dashboards-20.2.0-794.ga5c 206 kB/s | 43 kB 00:00 2026-02-22T20:32:33.456 INFO:teuthology.orchestra.run.trial084.stdout:(3/12): python3-jmespath-1.0.1-8.el10.noarch.rp 1.4 MB/s | 60 kB 00:00 2026-02-22T20:32:33.466 INFO:teuthology.orchestra.run.trial084.stdout:(4/12): python3-protobuf-3.19.6-11.el10.x86_64. 18 MB/s | 419 kB 00:00 2026-02-22T20:32:33.481 INFO:teuthology.orchestra.run.trial084.stdout:(5/12): protobuf-compiler-3.19.6-11.el10.x86_64 30 MB/s | 787 kB 00:00 2026-02-22T20:32:33.668 INFO:teuthology.orchestra.run.trial084.stdout:(6/12): grpc-data-1.48.4-44.el10_0.noarch.rpm 104 kB/s | 21 kB 00:00 2026-02-22T20:32:33.895 INFO:teuthology.orchestra.run.trial084.stdout:(7/12): python3-grpcio-tools-1.48.4-44.el10_0.x 693 kB/s | 157 kB 00:00 2026-02-22T20:32:33.965 INFO:teuthology.orchestra.run.trial084.stdout:(8/12): python3-repoze-lru-0.7-23.el10_0.noarch 516 kB/s | 36 kB 00:00 2026-02-22T20:32:34.076 INFO:teuthology.orchestra.run.trial084.stdout:(9/12): python3-routes-2.5.1-11.el10_0.noarch.r 1.8 MB/s | 201 kB 00:00 2026-02-22T20:32:34.120 INFO:teuthology.orchestra.run.trial084.stdout:(10/12): ceph-mgr-dashboard-20.2.0-794.ga5c3084 12 MB/s | 11 MB 00:00 2026-02-22T20:32:34.142 INFO:teuthology.orchestra.run.trial084.stdout:(11/12): python3-xmltodict-0.13.0-5.el10_0.noar 393 kB/s | 26 kB 00:00 2026-02-22T20:32:34.251 INFO:teuthology.orchestra.run.trial084.stdout:(12/12): python3-grpcio-1.48.4-44.el10_0.x86_64 3.0 MB/s | 2.3 MB 00:00 2026-02-22T20:32:34.252 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:34.252 INFO:teuthology.orchestra.run.trial084.stdout:Total 13 MB/s | 15 MB 00:01 2026-02-22T20:32:34.252 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:34.264 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:34.265 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:34.353 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:34.353 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:34.457 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:34.459 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-xmltodict-0.13.0-5.el10_0.noarch 1/12 2026-02-22T20:32:34.464 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-repoze-lru-0.7-23.el10_0.noarch 2/12 2026-02-22T20:32:34.466 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-routes-2.5.1-11.el10_0.noarch 3/12 2026-02-22T20:32:34.490 INFO:teuthology.orchestra.run.trial084.stdout: Installing : grpc-data-1.48.4-44.el10_0.noarch 4/12 2026-02-22T20:32:34.498 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-grpcio-1.48.4-44.el10_0.x86_64 5/12 2026-02-22T20:32:34.510 INFO:teuthology.orchestra.run.trial084.stdout: Installing : protobuf-compiler-3.19.6-11.el10.x86_64 6/12 2026-02-22T20:32:34.515 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-protobuf-3.19.6-11.el10.x86_64 7/12 2026-02-22T20:32:34.518 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-grpcio-tools-1.48.4-44.el10_0.x86_64 8/12 2026-02-22T20:32:34.520 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-jmespath-1.0.1-8.el10.noarch 9/12 2026-02-22T20:32:34.526 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-prometheus-alerts-2:20.2.0-794.ga5c3084f.el10 10/12 2026-02-22T20:32:35.209 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-grafana-dashboards-2:20.2.0-794.ga5c3084f.el1 11/12 2026-02-22T20:32:35.213 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-mgr-dashboard-2:20.2.0-794.ga5c3084f.el10.noa 12/12 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.2.0-794.ga5c3084f.el10.noa 12/12 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: ceph-grafana-dashboards-2:20.2.0-794.ga5c3084f.el10.noarch 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mgr-dashboard-2:20.2.0-794.ga5c3084f.el10.noarch 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: ceph-prometheus-alerts-2:20.2.0-794.ga5c3084f.el10.noarch 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: grpc-data-1.48.4-44.el10_0.noarch 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: protobuf-compiler-3.19.6-11.el10.x86_64 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: python3-grpcio-1.48.4-44.el10_0.x86_64 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: python3-grpcio-tools-1.48.4-44.el10_0.x86_64 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: python3-jmespath-1.0.1-8.el10.noarch 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: python3-protobuf-3.19.6-11.el10.x86_64 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: python3-repoze-lru-0.7-23.el10_0.noarch 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: python3-routes-2.5.1-11.el10_0.noarch 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: python3-xmltodict-0.13.0-5.el10_0.noarch 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:35.478 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:35.508 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-22T20:32:35.798 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:47 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:35.834 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:35.835 INFO:teuthology.orchestra.run.trial084.stdout:======================================================================================= 2026-02-22T20:32:35.835 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repository Size 2026-02-22T20:32:35.836 INFO:teuthology.orchestra.run.trial084.stdout:======================================================================================= 2026-02-22T20:32:35.836 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:35.836 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mgr-diskprediction-local noarch 2:20.2.0-794.ga5c3084f.el10 ceph-noarch 7.4 M 2026-02-22T20:32:35.836 INFO:teuthology.orchestra.run.trial084.stdout:Installing dependencies: 2026-02-22T20:32:35.836 INFO:teuthology.orchestra.run.trial084.stdout: flexiblas x86_64 3.4.2-3.el10 appstream 25 k 2026-02-22T20:32:35.836 INFO:teuthology.orchestra.run.trial084.stdout: flexiblas-netlib x86_64 3.4.2-3.el10 appstream 3.2 M 2026-02-22T20:32:35.836 INFO:teuthology.orchestra.run.trial084.stdout: flexiblas-openblas-openmp x86_64 3.4.2-3.el10 appstream 17 k 2026-02-22T20:32:35.836 INFO:teuthology.orchestra.run.trial084.stdout: libgfortran x86_64 14.3.1-2.1.el10 baseos 827 k 2026-02-22T20:32:35.836 INFO:teuthology.orchestra.run.trial084.stdout: libquadmath x86_64 14.3.1-2.1.el10 baseos 215 k 2026-02-22T20:32:35.836 INFO:teuthology.orchestra.run.trial084.stdout: openblas x86_64 0.3.29-2.el10 appstream 43 k 2026-02-22T20:32:35.836 INFO:teuthology.orchestra.run.trial084.stdout: openblas-openmp x86_64 0.3.29-2.el10 appstream 5.3 M 2026-02-22T20:32:35.836 INFO:teuthology.orchestra.run.trial084.stdout: python3-devel x86_64 3.12.12-3.el10_1 appstream 329 k 2026-02-22T20:32:35.837 INFO:teuthology.orchestra.run.trial084.stdout: python3-numpy x86_64 1:1.26.4-5.el10 appstream 6.7 M 2026-02-22T20:32:35.837 INFO:teuthology.orchestra.run.trial084.stdout: python3-numpy-f2py x86_64 1:1.26.4-5.el10 appstream 490 k 2026-02-22T20:32:35.837 INFO:teuthology.orchestra.run.trial084.stdout: python3-scipy x86_64 1.11.3-17.el10 appstream 16 M 2026-02-22T20:32:35.837 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:35.837 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:35.837 INFO:teuthology.orchestra.run.trial084.stdout:======================================================================================= 2026-02-22T20:32:35.837 INFO:teuthology.orchestra.run.trial084.stdout:Install 12 Packages 2026-02-22T20:32:35.837 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:35.837 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 41 M 2026-02-22T20:32:35.837 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 226 M 2026-02-22T20:32:35.837 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:35.873 INFO:teuthology.orchestra.run.trial084.stdout:(1/12): flexiblas-3.4.2-3.el10.x86_64.rpm 662 kB/s | 25 kB 00:00 2026-02-22T20:32:35.887 INFO:teuthology.orchestra.run.trial084.stdout:(2/12): flexiblas-openblas-openmp-3.4.2-3.el10. 1.2 MB/s | 17 kB 00:00 2026-02-22T20:32:35.903 INFO:teuthology.orchestra.run.trial084.stdout:(3/12): openblas-0.3.29-2.el10.x86_64.rpm 2.7 MB/s | 43 kB 00:00 2026-02-22T20:32:35.911 INFO:teuthology.orchestra.run.trial084.stdout:(4/12): flexiblas-netlib-3.4.2-3.el10.x86_64.rp 43 MB/s | 3.2 MB 00:00 2026-02-22T20:32:35.935 INFO:teuthology.orchestra.run.trial084.stdout:(5/12): python3-devel-3.12.12-3.el10_1.x86_64.r 13 MB/s | 329 kB 00:00 2026-02-22T20:32:35.998 INFO:teuthology.orchestra.run.trial084.stdout:(6/12): openblas-openmp-0.3.29-2.el10.x86_64.rp 56 MB/s | 5.3 MB 00:00 2026-02-22T20:32:36.026 INFO:teuthology.orchestra.run.trial084.stdout:(7/12): python3-numpy-f2py-1.26.4-5.el10.x86_64 17 MB/s | 490 kB 00:00 2026-02-22T20:32:36.043 INFO:teuthology.orchestra.run.trial084.stdout:(8/12): python3-numpy-1.26.4-5.el10.x86_64.rpm 62 MB/s | 6.7 MB 00:00 2026-02-22T20:32:36.060 INFO:teuthology.orchestra.run.trial084.stdout:(9/12): ceph-mgr-diskprediction-local-20.2.0-79 33 MB/s | 7.4 MB 00:00 2026-02-22T20:32:36.074 INFO:teuthology.orchestra.run.trial084.stdout:(10/12): libgfortran-14.3.1-2.1.el10.x86_64.rpm 27 MB/s | 827 kB 00:00 2026-02-22T20:32:36.082 INFO:teuthology.orchestra.run.trial084.stdout:(11/12): libquadmath-14.3.1-2.1.el10.x86_64.rpm 9.9 MB/s | 215 kB 00:00 2026-02-22T20:32:36.248 INFO:teuthology.orchestra.run.trial084.stdout:(12/12): python3-scipy-1.11.3-17.el10.x86_64.rp 73 MB/s | 16 MB 00:00 2026-02-22T20:32:36.249 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:36.249 INFO:teuthology.orchestra.run.trial084.stdout:Total 98 MB/s | 41 MB 00:00 2026-02-22T20:32:36.249 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:36.262 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:36.262 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:36.440 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:36.440 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:36.637 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:36.640 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libgfortran-14.3.1-2.1.el10.x86_64 1/12 2026-02-22T20:32:36.654 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libquadmath-14.3.1-2.1.el10.x86_64 2/12 2026-02-22T20:32:36.657 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-devel-3.12.12-3.el10_1.x86_64 3/12 2026-02-22T20:32:36.714 INFO:teuthology.orchestra.run.trial084.stdout: Installing : openblas-0.3.29-2.el10.x86_64 4/12 2026-02-22T20:32:36.734 INFO:teuthology.orchestra.run.trial084.stdout: Installing : openblas-openmp-0.3.29-2.el10.x86_64 5/12 2026-02-22T20:32:36.736 INFO:teuthology.orchestra.run.trial084.stdout: Installing : flexiblas-netlib-3.4.2-3.el10.x86_64 6/12 2026-02-22T20:32:36.737 INFO:teuthology.orchestra.run.trial084.stdout: Installing : flexiblas-openblas-openmp-3.4.2-3.el10.x86_64 7/12 2026-02-22T20:32:36.861 INFO:teuthology.orchestra.run.trial084.stdout: Installing : flexiblas-3.4.2-3.el10.x86_64 8/12 2026-02-22T20:32:36.887 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-numpy-1:1.26.4-5.el10.x86_64 9/12 2026-02-22T20:32:37.074 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-numpy-f2py-1:1.26.4-5.el10.x86_64 10/12 2026-02-22T20:32:37.429 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-scipy-1.11.3-17.el10.x86_64 11/12 2026-02-22T20:32:37.429 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-mgr-diskprediction-local-2:20.2.0-794.ga5c308 12/12 2026-02-22T20:32:37.722 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.2.0-794.ga5c308 12/12 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mgr-diskprediction-local-2:20.2.0-794.ga5c3084f.el10.noarch 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: flexiblas-3.4.2-3.el10.x86_64 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: flexiblas-netlib-3.4.2-3.el10.x86_64 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: flexiblas-openblas-openmp-3.4.2-3.el10.x86_64 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: libgfortran-14.3.1-2.1.el10.x86_64 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: libquadmath-14.3.1-2.1.el10.x86_64 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: openblas-0.3.29-2.el10.x86_64 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: openblas-openmp-0.3.29-2.el10.x86_64 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: python3-devel-3.12.12-3.el10_1.x86_64 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: python3-numpy-1:1.26.4-5.el10.x86_64 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: python3-numpy-f2py-1:1.26.4-5.el10.x86_64 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: python3-scipy-1.11.3-17.el10.x86_64 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:37.723 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:37.758 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install ceph-mgr-rook 2026-02-22T20:32:38.047 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:50 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:38.083 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:38.083 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:38.083 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repository Size 2026-02-22T20:32:38.083 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:38.083 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mgr-rook noarch 2:20.2.0-794.ga5c3084f.el10 ceph-noarch 50 k 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout:Installing dependencies: 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: python3-cachetools noarch 5.4.0-6.el10_0 epel 37 k 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: python3-certifi noarch 2023.05.07-6.el10_0 epel 14 k 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: python3-durationpy noarch 0.9-2.el10_0 epel 15 k 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: python3-google-auth noarch 1:2.40.3-1.el10_1 epel 286 k 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: python3-kubernetes noarch 1:32.0.1-1.el10_1 epel 1.3 M 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: python3-pyasn1 noarch 0.6.1-1.el10 appstream 198 k 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: python3-pyasn1-modules noarch 0.6.1-1.el10 appstream 402 k 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: python3-requests-oauthlib noarch 1.3.1-11.el10 appstream 60 k 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: python3-rsa noarch 4.9-7.el10_0 epel 70 k 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: python3-websocket-client noarch 1.8.0-2.el10_0 epel 124 k 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout:Install 11 Packages 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:38.084 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 2.6 M 2026-02-22T20:32:38.085 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 33 M 2026-02-22T20:32:38.085 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:38.262 INFO:teuthology.orchestra.run.trial084.stdout:(1/11): python3-pyasn1-modules-0.6.1-1.el10.noa 9.6 MB/s | 402 kB 00:00 2026-02-22T20:32:38.263 INFO:teuthology.orchestra.run.trial084.stdout:(2/11): python3-pyasn1-0.6.1-1.el10.noarch.rpm 4.7 MB/s | 198 kB 00:00 2026-02-22T20:32:38.275 INFO:teuthology.orchestra.run.trial084.stdout:(3/11): python3-requests-oauthlib-1.3.1-11.el10 4.7 MB/s | 60 kB 00:00 2026-02-22T20:32:38.295 INFO:teuthology.orchestra.run.trial084.stdout:(4/11): ceph-mgr-rook-20.2.0-794.ga5c3084f.el10 679 kB/s | 50 kB 00:00 2026-02-22T20:32:38.409 INFO:teuthology.orchestra.run.trial084.stdout:(5/11): python3-certifi-2023.05.07-6.el10_0.noa 107 kB/s | 14 kB 00:00 2026-02-22T20:32:38.431 INFO:teuthology.orchestra.run.trial084.stdout:(6/11): python3-durationpy-0.9-2.el10_0.noarch. 108 kB/s | 15 kB 00:00 2026-02-22T20:32:38.460 INFO:teuthology.orchestra.run.trial084.stdout:(7/11): python3-cachetools-5.4.0-6.el10_0.noarc 187 kB/s | 37 kB 00:00 2026-02-22T20:32:38.548 INFO:teuthology.orchestra.run.trial084.stdout:(8/11): python3-rsa-4.9-7.el10_0.noarch.rpm 797 kB/s | 70 kB 00:00 2026-02-22T20:32:38.588 INFO:teuthology.orchestra.run.trial084.stdout:(9/11): python3-google-auth-2.40.3-1.el10_1.noa 1.6 MB/s | 286 kB 00:00 2026-02-22T20:32:38.631 INFO:teuthology.orchestra.run.trial084.stdout:(10/11): python3-websocket-client-1.8.0-2.el10_ 1.5 MB/s | 124 kB 00:00 2026-02-22T20:32:38.697 INFO:teuthology.orchestra.run.trial084.stdout:(11/11): python3-kubernetes-32.0.1-1.el10_1.noa 5.0 MB/s | 1.3 MB 00:00 2026-02-22T20:32:38.697 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:38.697 INFO:teuthology.orchestra.run.trial084.stdout:Total 4.2 MB/s | 2.6 MB 00:00 2026-02-22T20:32:38.697 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:38.704 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:38.704 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:38.746 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:38.746 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:38.810 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:38.832 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-pyasn1-0.6.1-1.el10.noarch 1/11 2026-02-22T20:32:38.838 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-pyasn1-modules-0.6.1-1.el10.noarch 2/11 2026-02-22T20:32:38.844 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-rsa-4.9-7.el10_0.noarch 3/11 2026-02-22T20:32:38.846 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-websocket-client-1.8.0-2.el10_0.noarch 4/11 2026-02-22T20:32:38.848 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-durationpy-0.9-2.el10_0.noarch 5/11 2026-02-22T20:32:38.850 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-certifi-2023.05.07-6.el10_0.noarch 6/11 2026-02-22T20:32:38.864 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-cachetools-5.4.0-6.el10_0.noarch 7/11 2026-02-22T20:32:38.870 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-google-auth-1:2.40.3-1.el10_1.noarch 8/11 2026-02-22T20:32:39.004 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-requests-oauthlib-1.3.1-11.el10.noarch 9/11 2026-02-22T20:32:39.014 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-kubernetes-1:32.0.1-1.el10_1.noarch 10/11 2026-02-22T20:32:39.015 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-mgr-rook-2:20.2.0-794.ga5c3084f.el10.noarch 11/11 2026-02-22T20:32:39.170 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-mgr-rook-2:20.2.0-794.ga5c3084f.el10.noarch 11/11 2026-02-22T20:32:39.170 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:39.170 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:39.170 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mgr-rook-2:20.2.0-794.ga5c3084f.el10.noarch 2026-02-22T20:32:39.170 INFO:teuthology.orchestra.run.trial084.stdout: python3-cachetools-5.4.0-6.el10_0.noarch 2026-02-22T20:32:39.171 INFO:teuthology.orchestra.run.trial084.stdout: python3-certifi-2023.05.07-6.el10_0.noarch 2026-02-22T20:32:39.171 INFO:teuthology.orchestra.run.trial084.stdout: python3-durationpy-0.9-2.el10_0.noarch 2026-02-22T20:32:39.171 INFO:teuthology.orchestra.run.trial084.stdout: python3-google-auth-1:2.40.3-1.el10_1.noarch 2026-02-22T20:32:39.171 INFO:teuthology.orchestra.run.trial084.stdout: python3-kubernetes-1:32.0.1-1.el10_1.noarch 2026-02-22T20:32:39.171 INFO:teuthology.orchestra.run.trial084.stdout: python3-pyasn1-0.6.1-1.el10.noarch 2026-02-22T20:32:39.171 INFO:teuthology.orchestra.run.trial084.stdout: python3-pyasn1-modules-0.6.1-1.el10.noarch 2026-02-22T20:32:39.171 INFO:teuthology.orchestra.run.trial084.stdout: python3-requests-oauthlib-1.3.1-11.el10.noarch 2026-02-22T20:32:39.171 INFO:teuthology.orchestra.run.trial084.stdout: python3-rsa-4.9-7.el10_0.noarch 2026-02-22T20:32:39.171 INFO:teuthology.orchestra.run.trial084.stdout: python3-websocket-client-1.8.0-2.el10_0.noarch 2026-02-22T20:32:39.171 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:39.171 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:39.199 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install ceph-mgr-cephadm 2026-02-22T20:32:39.484 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:51 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:39.522 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repository Size 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mgr-cephadm noarch 2:20.2.0-794.ga5c3084f.el10 ceph-noarch 179 k 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout:Installing dependencies: 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout: python3-asyncssh noarch 2.17.0-1.el10_0 epel 756 k 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout: python3-jinja2 noarch 3.1.6-1.el10_0 appstream 330 k 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout: python3-markupsafe x86_64 2.1.3-6.el10 appstream 34 k 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout: python3-natsort noarch 8.4.0-2.el10_0 epel 72 k 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout:Install 5 Packages 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:39.523 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 1.3 M 2026-02-22T20:32:39.524 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 7.9 M 2026-02-22T20:32:39.524 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:39.686 INFO:teuthology.orchestra.run.trial084.stdout:(1/5): python3-markupsafe-2.1.3-6.el10.x86_64.r 895 kB/s | 34 kB 00:00 2026-02-22T20:32:39.686 INFO:teuthology.orchestra.run.trial084.stdout:(2/5): python3-jinja2-3.1.6-1.el10_0.noarch.rpm 8.2 MB/s | 330 kB 00:00 2026-02-22T20:32:39.710 INFO:teuthology.orchestra.run.trial084.stdout:(3/5): python3-natsort-8.4.0-2.el10_0.noarch.rp 2.9 MB/s | 72 kB 00:00 2026-02-22T20:32:39.719 INFO:teuthology.orchestra.run.trial084.stdout:(4/5): python3-asyncssh-2.17.0-1.el10_0.noarch. 22 MB/s | 756 kB 00:00 2026-02-22T20:32:39.730 INFO:teuthology.orchestra.run.trial084.stdout:(5/5): ceph-mgr-cephadm-20.2.0-794.ga5c3084f.el 2.1 MB/s | 179 kB 00:00 2026-02-22T20:32:39.731 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:39.731 INFO:teuthology.orchestra.run.trial084.stdout:Total 6.4 MB/s | 1.3 MB 00:00 2026-02-22T20:32:39.731 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:39.736 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:39.736 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:39.782 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:39.783 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:39.849 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:39.870 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-natsort-8.4.0-2.el10_0.noarch 1/5 2026-02-22T20:32:39.874 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-asyncssh-2.17.0-1.el10_0.noarch 2/5 2026-02-22T20:32:39.882 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-markupsafe-2.1.3-6.el10.x86_64 3/5 2026-02-22T20:32:39.894 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-jinja2-3.1.6-1.el10_0.noarch 4/5 2026-02-22T20:32:39.895 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-mgr-cephadm-2:20.2.0-794.ga5c3084f.el10.noarch 5/5 2026-02-22T20:32:40.024 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.2.0-794.ga5c3084f.el10.noarch 5/5 2026-02-22T20:32:40.024 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:40.024 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:40.024 INFO:teuthology.orchestra.run.trial084.stdout: ceph-mgr-cephadm-2:20.2.0-794.ga5c3084f.el10.noarch 2026-02-22T20:32:40.025 INFO:teuthology.orchestra.run.trial084.stdout: python3-asyncssh-2.17.0-1.el10_0.noarch 2026-02-22T20:32:40.025 INFO:teuthology.orchestra.run.trial084.stdout: python3-jinja2-3.1.6-1.el10_0.noarch 2026-02-22T20:32:40.025 INFO:teuthology.orchestra.run.trial084.stdout: python3-markupsafe-2.1.3-6.el10.x86_64 2026-02-22T20:32:40.025 INFO:teuthology.orchestra.run.trial084.stdout: python3-natsort-8.4.0-2.el10_0.noarch 2026-02-22T20:32:40.025 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:40.025 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:40.051 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install ceph-fuse 2026-02-22T20:32:40.336 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:52 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:40.375 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:40.375 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:40.375 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repo Size 2026-02-22T20:32:40.375 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:40.375 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:40.376 INFO:teuthology.orchestra.run.trial084.stdout: ceph-fuse x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 1.0 M 2026-02-22T20:32:40.376 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:40.376 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:40.376 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:40.376 INFO:teuthology.orchestra.run.trial084.stdout:Install 1 Package 2026-02-22T20:32:40.376 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:40.376 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 1.0 M 2026-02-22T20:32:40.376 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 2.9 M 2026-02-22T20:32:40.376 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:40.502 INFO:teuthology.orchestra.run.trial084.stdout:ceph-fuse-20.2.0-794.ga5c3084f.el10.x86_64.rpm 7.9 MB/s | 1.0 MB 00:00 2026-02-22T20:32:40.502 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:40.502 INFO:teuthology.orchestra.run.trial084.stdout:Total 7.8 MB/s | 1.0 MB 00:00 2026-02-22T20:32:40.502 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:40.507 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:40.507 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:40.538 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:40.538 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:40.616 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:40.634 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-fuse-2:20.2.0-794.ga5c3084f.el10.x86_64 1/1 2026-02-22T20:32:40.881 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-fuse-2:20.2.0-794.ga5c3084f.el10.x86_64 1/1 2026-02-22T20:32:40.881 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:40.881 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:40.881 INFO:teuthology.orchestra.run.trial084.stdout: ceph-fuse-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:40.881 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:40.881 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:40.905 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install ceph-volume 2026-02-22T20:32:41.197 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:53 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:41.232 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repository Size 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout: ceph-volume noarch 2:20.2.0-794.ga5c3084f.el10 ceph-noarch 375 k 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout:Installing dependencies: 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout: cryptsetup x86_64 2.7.5-2.el10 baseos 335 k 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout:Install 2 Packages 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 710 k 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 2.5 M 2026-02-22T20:32:41.233 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:41.278 INFO:teuthology.orchestra.run.trial084.stdout:(1/2): cryptsetup-2.7.5-2.el10.x86_64.rpm 7.5 MB/s | 335 kB 00:00 2026-02-22T20:32:41.327 INFO:teuthology.orchestra.run.trial084.stdout:(2/2): ceph-volume-20.2.0-794.ga5c3084f.el10.no 3.9 MB/s | 375 kB 00:00 2026-02-22T20:32:41.328 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:41.328 INFO:teuthology.orchestra.run.trial084.stdout:Total 7.3 MB/s | 710 kB 00:00 2026-02-22T20:32:41.328 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:41.335 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:41.335 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:41.377 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:41.377 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:41.441 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:41.472 INFO:teuthology.orchestra.run.trial084.stdout: Installing : cryptsetup-2.7.5-2.el10.x86_64 1/2 2026-02-22T20:32:41.474 INFO:teuthology.orchestra.run.trial084.stdout: Installing : ceph-volume-2:20.2.0-794.ga5c3084f.el10.noarch 2/2 2026-02-22T20:32:41.490 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: ceph-volume-2:20.2.0-794.ga5c3084f.el10.noarch 2/2 2026-02-22T20:32:41.490 INFO:teuthology.orchestra.run.trial084.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-22T20:32:41.490 INFO:teuthology.orchestra.run.trial084.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-22T20:32:41.490 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:41.770 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:41.770 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:41.770 INFO:teuthology.orchestra.run.trial084.stdout: ceph-volume-2:20.2.0-794.ga5c3084f.el10.noarch cryptsetup-2.7.5-2.el10.x86_64 2026-02-22T20:32:41.770 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:41.770 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:41.796 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install librados-devel 2026-02-22T20:32:42.086 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:54 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:42.124 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:42.124 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:42.124 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repo Size 2026-02-22T20:32:42.124 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:42.124 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:42.124 INFO:teuthology.orchestra.run.trial084.stdout: librados-devel x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 128 k 2026-02-22T20:32:42.124 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:42.124 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:42.124 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:42.125 INFO:teuthology.orchestra.run.trial084.stdout:Install 1 Package 2026-02-22T20:32:42.125 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:42.125 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 128 k 2026-02-22T20:32:42.125 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 448 k 2026-02-22T20:32:42.125 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:42.206 INFO:teuthology.orchestra.run.trial084.stdout:librados-devel-20.2.0-794.ga5c3084f.el10.x86_64 1.5 MB/s | 128 kB 00:00 2026-02-22T20:32:42.207 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:42.207 INFO:teuthology.orchestra.run.trial084.stdout:Total 1.5 MB/s | 128 kB 00:00 2026-02-22T20:32:42.207 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:42.210 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:42.210 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:42.238 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:42.238 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:42.288 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:42.306 INFO:teuthology.orchestra.run.trial084.stdout: Installing : librados-devel-2:20.2.0-794.ga5c3084f.el10.x86_64 1/1 2026-02-22T20:32:42.416 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: librados-devel-2:20.2.0-794.ga5c3084f.el10.x86_64 1/1 2026-02-22T20:32:42.417 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:42.417 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:42.417 INFO:teuthology.orchestra.run.trial084.stdout: librados-devel-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:42.417 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:42.417 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:42.441 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install libcephfs2 2026-02-22T20:32:42.724 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:54 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:42.751 INFO:teuthology.orchestra.run.trial084.stdout:Package libcephfs2-2:20.2.0-794.ga5c3084f.el10.x86_64 is already installed. 2026-02-22T20:32:42.760 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:42.761 INFO:teuthology.orchestra.run.trial084.stdout:Nothing to do. 2026-02-22T20:32:42.761 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:42.781 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install libcephfs-devel 2026-02-22T20:32:43.067 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:55 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:43.103 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repo Size 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout: libcephfs-devel x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 34 k 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout:Installing dependencies: 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout: libcephfs-proxy2 x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 25 k 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout:Install 2 Packages 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 59 k 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 208 k 2026-02-22T20:32:43.104 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:43.166 INFO:teuthology.orchestra.run.trial084.stdout:(1/2): libcephfs-devel-20.2.0-794.ga5c3084f.el1 568 kB/s | 34 kB 00:00 2026-02-22T20:32:43.168 INFO:teuthology.orchestra.run.trial084.stdout:(2/2): libcephfs-proxy2-20.2.0-794.ga5c3084f.el 395 kB/s | 25 kB 00:00 2026-02-22T20:32:43.168 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:43.169 INFO:teuthology.orchestra.run.trial084.stdout:Total 929 kB/s | 59 kB 00:00 2026-02-22T20:32:43.169 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:43.171 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:43.171 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:43.201 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:43.201 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:43.247 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:43.248 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libcephfs-proxy2-2:20.2.0-794.ga5c3084f.el10.x86_64 1/2 2026-02-22T20:32:43.262 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: libcephfs-proxy2-2:20.2.0-794.ga5c3084f.el10.x86_64 1/2 2026-02-22T20:32:43.281 INFO:teuthology.orchestra.run.trial084.stdout: Installing : libcephfs-devel-2:20.2.0-794.ga5c3084f.el10.x86_64 2/2 2026-02-22T20:32:43.378 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: libcephfs-devel-2:20.2.0-794.ga5c3084f.el10.x86_64 2/2 2026-02-22T20:32:43.378 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:43.378 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:43.378 INFO:teuthology.orchestra.run.trial084.stdout: libcephfs-devel-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:43.378 INFO:teuthology.orchestra.run.trial084.stdout: libcephfs-proxy2-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:43.378 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:43.378 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:43.401 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install librados2 2026-02-22T20:32:43.687 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:55 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:43.715 INFO:teuthology.orchestra.run.trial084.stdout:Package librados2-2:20.2.0-794.ga5c3084f.el10.x86_64 is already installed. 2026-02-22T20:32:43.725 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:43.725 INFO:teuthology.orchestra.run.trial084.stdout:Nothing to do. 2026-02-22T20:32:43.726 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:43.744 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install librbd1 2026-02-22T20:32:44.033 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:56 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:44.060 INFO:teuthology.orchestra.run.trial084.stdout:Package librbd1-2:20.2.0-794.ga5c3084f.el10.x86_64 is already installed. 2026-02-22T20:32:44.070 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:44.071 INFO:teuthology.orchestra.run.trial084.stdout:Nothing to do. 2026-02-22T20:32:44.071 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:44.091 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install python3-rados 2026-02-22T20:32:44.371 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:56 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:44.399 INFO:teuthology.orchestra.run.trial084.stdout:Package python3-rados-2:20.2.0-794.ga5c3084f.el10.x86_64 is already installed. 2026-02-22T20:32:44.408 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:44.409 INFO:teuthology.orchestra.run.trial084.stdout:Nothing to do. 2026-02-22T20:32:44.409 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:44.430 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install python3-rgw 2026-02-22T20:32:44.713 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:56 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:44.740 INFO:teuthology.orchestra.run.trial084.stdout:Package python3-rgw-2:20.2.0-794.ga5c3084f.el10.x86_64 is already installed. 2026-02-22T20:32:44.750 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:44.750 INFO:teuthology.orchestra.run.trial084.stdout:Nothing to do. 2026-02-22T20:32:44.750 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:44.770 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install python3-cephfs 2026-02-22T20:32:45.053 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:57 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:45.078 INFO:teuthology.orchestra.run.trial084.stdout:Package python3-cephfs-2:20.2.0-794.ga5c3084f.el10.x86_64 is already installed. 2026-02-22T20:32:45.088 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:45.088 INFO:teuthology.orchestra.run.trial084.stdout:Nothing to do. 2026-02-22T20:32:45.089 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:45.109 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install python3-rbd 2026-02-22T20:32:45.396 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:57 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:45.422 INFO:teuthology.orchestra.run.trial084.stdout:Package python3-rbd-2:20.2.0-794.ga5c3084f.el10.x86_64 is already installed. 2026-02-22T20:32:45.432 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:45.433 INFO:teuthology.orchestra.run.trial084.stdout:Nothing to do. 2026-02-22T20:32:45.433 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:45.452 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install rbd-fuse 2026-02-22T20:32:45.746 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:57 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:45.784 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:45.785 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:45.785 INFO:teuthology.orchestra.run.trial084.stdout: Package Architecture Version Repository Size 2026-02-22T20:32:45.785 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:45.785 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:45.785 INFO:teuthology.orchestra.run.trial084.stdout: rbd-fuse x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 93 k 2026-02-22T20:32:45.785 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:45.785 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:45.785 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:45.785 INFO:teuthology.orchestra.run.trial084.stdout:Install 1 Package 2026-02-22T20:32:45.785 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:45.785 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 93 k 2026-02-22T20:32:45.786 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 228 k 2026-02-22T20:32:45.786 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:45.867 INFO:teuthology.orchestra.run.trial084.stdout:rbd-fuse-20.2.0-794.ga5c3084f.el10.x86_64.rpm 1.1 MB/s | 93 kB 00:00 2026-02-22T20:32:45.868 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:45.868 INFO:teuthology.orchestra.run.trial084.stdout:Total 1.1 MB/s | 93 kB 00:00 2026-02-22T20:32:45.868 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:45.872 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:45.872 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:45.900 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:45.900 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:45.948 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:45.966 INFO:teuthology.orchestra.run.trial084.stdout: Installing : rbd-fuse-2:20.2.0-794.ga5c3084f.el10.x86_64 1/1 2026-02-22T20:32:46.079 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: rbd-fuse-2:20.2.0-794.ga5c3084f.el10.x86_64 1/1 2026-02-22T20:32:46.079 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:46.079 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:46.079 INFO:teuthology.orchestra.run.trial084.stdout: rbd-fuse-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:46.080 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:46.080 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:46.105 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install rbd-mirror 2026-02-22T20:32:46.399 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:58 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:46.435 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repo Size 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout: rbd-mirror x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 2.9 M 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout:Install 1 Package 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 2.9 M 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 11 M 2026-02-22T20:32:46.436 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:46.582 INFO:teuthology.orchestra.run.trial084.stdout:rbd-mirror-20.2.0-794.ga5c3084f.el10.x86_64.rpm 20 MB/s | 2.9 MB 00:00 2026-02-22T20:32:46.582 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:46.582 INFO:teuthology.orchestra.run.trial084.stdout:Total 20 MB/s | 2.9 MB 00:00 2026-02-22T20:32:46.582 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:46.587 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:46.587 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:46.624 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:46.624 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:46.770 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:46.771 INFO:teuthology.orchestra.run.trial084.stdout: Installing : rbd-mirror-2:20.2.0-794.ga5c3084f.el10.x86_64 1/1 2026-02-22T20:32:46.794 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: rbd-mirror-2:20.2.0-794.ga5c3084f.el10.x86_64 1/1 2026-02-22T20:32:46.794 INFO:teuthology.orchestra.run.trial084.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-22T20:32:46.794 INFO:teuthology.orchestra.run.trial084.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-22T20:32:46.794 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target' → '/usr/lib/systemd/system/ceph-rbd-mirror.target'. 2026-02-22T20:32:46.794 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target' → '/usr/lib/systemd/system/ceph-rbd-mirror.target'. 2026-02-22T20:32:46.794 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:47.073 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:47.073 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:47.073 INFO:teuthology.orchestra.run.trial084.stdout: rbd-mirror-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:47.073 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:47.073 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:47.101 DEBUG:teuthology.orchestra.run.trial084:> sudo yum -y install rbd-nbd 2026-02-22T20:32:47.393 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:00:59 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:32:47.431 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:32:47.432 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:47.432 INFO:teuthology.orchestra.run.trial084.stdout: Package Architecture Version Repository Size 2026-02-22T20:32:47.432 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:47.432 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:32:47.432 INFO:teuthology.orchestra.run.trial084.stdout: rbd-nbd x86_64 2:20.2.0-794.ga5c3084f.el10 ceph 184 k 2026-02-22T20:32:47.432 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:47.432 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:32:47.432 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:32:47.432 INFO:teuthology.orchestra.run.trial084.stdout:Install 1 Package 2026-02-22T20:32:47.433 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:47.433 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 184 k 2026-02-22T20:32:47.433 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 492 k 2026-02-22T20:32:47.433 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:32:47.514 INFO:teuthology.orchestra.run.trial084.stdout:rbd-nbd-20.2.0-794.ga5c3084f.el10.x86_64.rpm 2.2 MB/s | 184 kB 00:00 2026-02-22T20:32:47.514 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:32:47.515 INFO:teuthology.orchestra.run.trial084.stdout:Total 2.2 MB/s | 184 kB 00:00 2026-02-22T20:32:47.515 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:32:47.519 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:32:47.519 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:32:47.547 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:32:47.548 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:32:47.599 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:32:47.617 INFO:teuthology.orchestra.run.trial084.stdout: Installing : rbd-nbd-2:20.2.0-794.ga5c3084f.el10.x86_64 1/1 2026-02-22T20:32:47.737 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: rbd-nbd-2:20.2.0-794.ga5c3084f.el10.x86_64 1/1 2026-02-22T20:32:47.737 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:47.737 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:32:47.737 INFO:teuthology.orchestra.run.trial084.stdout: rbd-nbd-2:20.2.0-794.ga5c3084f.el10.x86_64 2026-02-22T20:32:47.738 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:47.738 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:32:47.768 DEBUG:teuthology.parallel:result is None 2026-02-22T20:32:47.769 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=rocky%2F10%2Fx86_64&sha1=a5c3084f688b1f6eff223b8d259a8316f9844c8e 2026-02-22T20:32:47.943 DEBUG:teuthology.orchestra.run.trial084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-22T20:32:47.964 INFO:teuthology.orchestra.run.trial084.stdout:20.2.0-794.ga5c3084f.el10 2026-02-22T20:32:47.965 INFO:teuthology.packaging:The installed version of ceph is 20.2.0-794.ga5c3084f.el10 2026-02-22T20:32:47.965 INFO:teuthology.task.install:The correct ceph version 20.2.0-794.ga5c3084f is installed. 2026-02-22T20:32:47.968 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-22T20:32:47.969 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:47.969 DEBUG:teuthology.orchestra.run.trial084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-22T20:32:48.032 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-22T20:32:48.032 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:48.032 DEBUG:teuthology.orchestra.run.trial084:> sudo dd of=/usr/bin/daemon-helper 2026-02-22T20:32:48.100 DEBUG:teuthology.orchestra.run.trial084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-22T20:32:48.167 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-22T20:32:48.167 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:48.168 DEBUG:teuthology.orchestra.run.trial084:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-22T20:32:48.233 DEBUG:teuthology.orchestra.run.trial084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-22T20:32:48.299 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-22T20:32:48.299 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:48.300 DEBUG:teuthology.orchestra.run.trial084:> sudo dd of=/usr/bin/stdin-killer 2026-02-22T20:32:48.364 DEBUG:teuthology.orchestra.run.trial084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-22T20:32:48.431 INFO:teuthology.run_tasks:Running task ceph... 2026-02-22T20:32:48.548 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-22T20:32:48.548 DEBUG:teuthology.orchestra.run.trial084:> sudo chmod 777 /var/log/ceph 2026-02-22T20:32:48.577 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-22T20:32:48.577 DEBUG:teuthology.orchestra.run.trial084:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-22T20:32:48.643 INFO:tasks.ceph:Creating extra log directories... 2026-02-22T20:32:48.644 DEBUG:teuthology.orchestra.run.trial084:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-22T20:32:48.718 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-22T20:32:48.718 INFO:tasks.ceph:config {'conf': {'osd': {'osd min pg log entries': 5, 'osd_fast_fail_on_connection_refused': False, 'bdev async discard': True, 'bdev enable discard': True, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore onode segment size': '1024K', 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-22T20:32:48.718 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/nmordech-2026-02-22_19:44:01-rados-wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle-distro-default-trial/64522', 'branch': 'wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle', 'description': 'rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async objectstore/{bluestore/{alloc$/{hybrid} base mem$/{low} onode-segment$/{1M} write$/{v2/{compr$/{no$/{no}} v2}}}} rados supported-random-distro$/{rpm_latest}}', 'email': None, 'first_in_suite': False, 'flavor': 'default', 'job_id': '64522', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'nmordech-2026-02-22_19:44:01-rados-wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle-distro-default-trial', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'rocky', 'os_version': '10.1', 'overrides': {'admin_socket': {'branch': 'wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore onode segment size': '1024K', 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'a5c3084f688b1f6eff223b8d259a8316f9844c8e'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'distro-suffix': 'rockylinux-10'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a5c3084f688b1f6eff223b8d259a8316f9844c8e'}}, 'selinux': {'allowlist': ['comm="systemd".*denied.*\\{ prog_run \\}.*tclass=bpf.*permissive=1']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle', 'sha1': 'a5c3084f688b1f6eff223b8d259a8316f9844c8e'}}, 'owner': 'scheduled_nmordech@soko04.front.sepia.ceph.com', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 966, 'sha1': 'a5c3084f688b1f6eff223b8d259a8316f9844c8e', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a5c3084f688b1f6eff223b8d259a8316f9844c8e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a5c3084f688b1f6eff223b8d259a8316f9844c8e', 'targets': {'trial084.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB7B7+qKjlR5tZuvVbl/t06SF9cyUM9elTViDNxRqMTJlt+ehCvw88ZNycOYou0wy6tA7AhFaj7ixMy0W0pk8J4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'osd min pg log entries': 5, 'osd_fast_fail_on_connection_refused': False, 'bdev async discard': True, 'bdev enable discard': True, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore onode segment size': '1024K', 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'log-ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'flavor': 'default', 'fs': 'xfs', 'sha1': 'a5c3084f688b1f6eff223b8d259a8316f9844c8e', 'cluster': 'ceph'}}, {'osd_recovery.test_incomplete_pgs': None}, {'pexec': {'all': ['sudo dnf remove nvme-cli -y', 'sudo dnf install nvmetcli nvme-cli -y']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-22_19:44:01', 'tube': 'trial', 'user': 'nmordech', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653'} 2026-02-22T20:32:48.718 DEBUG:teuthology.orchestra.run.trial084:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-22T20:32:48.769 DEBUG:teuthology.orchestra.run.trial084:> sudo install -d -m0777 -- /var/run/ceph 2026-02-22T20:32:48.835 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:48.835 DEBUG:teuthology.orchestra.run.trial084:> dd if=/scratch_devs of=/dev/stdout 2026-02-22T20:32:48.892 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-22T20:32:48.892 DEBUG:teuthology.orchestra.run.trial084:> stat /dev/vg_nvme/lv_1 2026-02-22T20:32:48.949 INFO:teuthology.orchestra.run.trial084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-22T20:32:48.949 INFO:teuthology.orchestra.run.trial084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T20:32:48.949 INFO:teuthology.orchestra.run.trial084.stdout:Device: 0,6 Inode: 884 Links: 1 2026-02-22T20:32:48.949 INFO:teuthology.orchestra.run.trial084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T20:32:48.949 INFO:teuthology.orchestra.run.trial084.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T20:32:48.949 INFO:teuthology.orchestra.run.trial084.stdout:Access: 2026-02-22 20:32:46.944221401 +0000 2026-02-22T20:32:48.949 INFO:teuthology.orchestra.run.trial084.stdout:Modify: 2026-02-22 20:31:34.985719738 +0000 2026-02-22T20:32:48.949 INFO:teuthology.orchestra.run.trial084.stdout:Change: 2026-02-22 20:31:34.985719738 +0000 2026-02-22T20:32:48.949 INFO:teuthology.orchestra.run.trial084.stdout: Birth: 2026-02-22 20:31:34.985719738 +0000 2026-02-22T20:32:48.949 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-22T20:32:49.015 INFO:teuthology.orchestra.run.trial084.stderr:1+0 records in 2026-02-22T20:32:49.015 INFO:teuthology.orchestra.run.trial084.stderr:1+0 records out 2026-02-22T20:32:49.015 INFO:teuthology.orchestra.run.trial084.stderr:512 bytes copied, 0.000131564 s, 3.9 MB/s 2026-02-22T20:32:49.017 DEBUG:teuthology.orchestra.run.trial084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-22T20:32:49.076 DEBUG:teuthology.orchestra.run.trial084:> stat /dev/vg_nvme/lv_2 2026-02-22T20:32:49.133 INFO:teuthology.orchestra.run.trial084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-22T20:32:49.134 INFO:teuthology.orchestra.run.trial084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T20:32:49.134 INFO:teuthology.orchestra.run.trial084.stdout:Device: 0,6 Inode: 892 Links: 1 2026-02-22T20:32:49.134 INFO:teuthology.orchestra.run.trial084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T20:32:49.134 INFO:teuthology.orchestra.run.trial084.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T20:32:49.134 INFO:teuthology.orchestra.run.trial084.stdout:Access: 2026-02-22 20:32:46.944221401 +0000 2026-02-22T20:32:49.134 INFO:teuthology.orchestra.run.trial084.stdout:Modify: 2026-02-22 20:31:35.214714234 +0000 2026-02-22T20:32:49.134 INFO:teuthology.orchestra.run.trial084.stdout:Change: 2026-02-22 20:31:35.214714234 +0000 2026-02-22T20:32:49.134 INFO:teuthology.orchestra.run.trial084.stdout: Birth: 2026-02-22 20:31:35.214714234 +0000 2026-02-22T20:32:49.134 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-22T20:32:49.198 INFO:teuthology.orchestra.run.trial084.stderr:1+0 records in 2026-02-22T20:32:49.198 INFO:teuthology.orchestra.run.trial084.stderr:1+0 records out 2026-02-22T20:32:49.198 INFO:teuthology.orchestra.run.trial084.stderr:512 bytes copied, 0.000128137 s, 4.0 MB/s 2026-02-22T20:32:49.199 DEBUG:teuthology.orchestra.run.trial084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-22T20:32:49.258 DEBUG:teuthology.orchestra.run.trial084:> stat /dev/vg_nvme/lv_3 2026-02-22T20:32:49.316 INFO:teuthology.orchestra.run.trial084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-22T20:32:49.316 INFO:teuthology.orchestra.run.trial084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T20:32:49.316 INFO:teuthology.orchestra.run.trial084.stdout:Device: 0,6 Inode: 905 Links: 1 2026-02-22T20:32:49.316 INFO:teuthology.orchestra.run.trial084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T20:32:49.316 INFO:teuthology.orchestra.run.trial084.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T20:32:49.316 INFO:teuthology.orchestra.run.trial084.stdout:Access: 2026-02-22 20:32:46.945221377 +0000 2026-02-22T20:32:49.316 INFO:teuthology.orchestra.run.trial084.stdout:Modify: 2026-02-22 20:31:35.534706541 +0000 2026-02-22T20:32:49.316 INFO:teuthology.orchestra.run.trial084.stdout:Change: 2026-02-22 20:31:35.534706541 +0000 2026-02-22T20:32:49.317 INFO:teuthology.orchestra.run.trial084.stdout: Birth: 2026-02-22 20:31:35.534706541 +0000 2026-02-22T20:32:49.317 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-22T20:32:49.381 INFO:teuthology.orchestra.run.trial084.stderr:1+0 records in 2026-02-22T20:32:49.381 INFO:teuthology.orchestra.run.trial084.stderr:1+0 records out 2026-02-22T20:32:49.381 INFO:teuthology.orchestra.run.trial084.stderr:512 bytes copied, 0.000147584 s, 3.5 MB/s 2026-02-22T20:32:49.383 DEBUG:teuthology.orchestra.run.trial084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-22T20:32:49.441 DEBUG:teuthology.orchestra.run.trial084:> stat /dev/vg_nvme/lv_4 2026-02-22T20:32:49.499 INFO:teuthology.orchestra.run.trial084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-22T20:32:49.499 INFO:teuthology.orchestra.run.trial084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T20:32:49.499 INFO:teuthology.orchestra.run.trial084.stdout:Device: 0,6 Inode: 918 Links: 1 2026-02-22T20:32:49.499 INFO:teuthology.orchestra.run.trial084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T20:32:49.499 INFO:teuthology.orchestra.run.trial084.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T20:32:49.499 INFO:teuthology.orchestra.run.trial084.stdout:Access: 2026-02-22 20:32:46.945221377 +0000 2026-02-22T20:32:49.499 INFO:teuthology.orchestra.run.trial084.stdout:Modify: 2026-02-22 20:31:35.831699402 +0000 2026-02-22T20:32:49.499 INFO:teuthology.orchestra.run.trial084.stdout:Change: 2026-02-22 20:31:35.831699402 +0000 2026-02-22T20:32:49.499 INFO:teuthology.orchestra.run.trial084.stdout: Birth: 2026-02-22 20:31:35.831699402 +0000 2026-02-22T20:32:49.499 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-22T20:32:49.564 INFO:teuthology.orchestra.run.trial084.stderr:1+0 records in 2026-02-22T20:32:49.564 INFO:teuthology.orchestra.run.trial084.stderr:1+0 records out 2026-02-22T20:32:49.564 INFO:teuthology.orchestra.run.trial084.stderr:512 bytes copied, 0.000128889 s, 4.0 MB/s 2026-02-22T20:32:49.566 DEBUG:teuthology.orchestra.run.trial084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-22T20:32:49.625 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-22T20:32:49.625 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial084.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2026-02-22T20:32:49.625 INFO:tasks.ceph:Generating config... 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] osd min pg log entries = 5 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] osd_fast_fail_on_connection_refused = False 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] bdev async discard = True 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] bdev enable discard = True 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] bluestore onode segment size = 1024K 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] bluestore write v2 = True 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] debug bluefs = 20 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] debug bluestore = 20 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] debug rocksdb = 10 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] osd mclock skip benchmark = True 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] osd op queue = debug_random 2026-02-22T20:32:49.626 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2026-02-22T20:32:49.627 INFO:tasks.ceph:[global] mon election default strategy = 1 2026-02-22T20:32:49.627 INFO:tasks.ceph:[global] ms type = async 2026-02-22T20:32:49.627 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-22T20:32:49.627 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-22T20:32:49.627 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-22T20:32:49.627 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-22T20:32:49.627 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-22T20:32:49.627 INFO:tasks.ceph:[mon] mon scrub interval = 300 2026-02-22T20:32:49.627 INFO:tasks.ceph:Setting up mon.a... 2026-02-22T20:32:49.627 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-22T20:32:49.701 INFO:teuthology.orchestra.run.trial084.stdout:creating /etc/ceph/ceph.keyring 2026-02-22T20:32:49.704 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-22T20:32:49.782 DEBUG:teuthology.orchestra.run.trial084:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T20:32:49.847 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.84'), ('mon.b', '[v2:10.20.193.84:3301,v1:10.20.193.84:6790]'), ('mon.c', '[v2:10.20.193.84:3302,v1:10.20.193.84:6791]')] 2026-02-22T20:32:49.847 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.84,[v2:10.20.193.84:3301,v1:10.20.193.84:6790],[v2:10.20.193.84:3302,v1:10.20.193.84:6791]', 'mon election default strategy': 1, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'osd min pg log entries': 5, 'osd_fast_fail_on_connection_refused': False, 'bdev async discard': True, 'bdev enable discard': True, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore onode segment size': '1024K', 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2026-02-22T20:32:49.848 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:49.848 DEBUG:teuthology.orchestra.run.trial084:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-22T20:32:49.904 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.84 --addv b '[v2:10.20.193.84:3301,v1:10.20.193.84:6790]' --addv c '[v2:10.20.193.84:3302,v1:10.20.193.84:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2026-02-22T20:32:49.978 INFO:teuthology.orchestra.run.trial084.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-22T20:32:49.979 INFO:teuthology.orchestra.run.trial084.stdout:monmaptool: generated fsid a4ea559c-fa00-404b-b21c-f34ee9bd0795 2026-02-22T20:32:49.979 INFO:teuthology.orchestra.run.trial084.stdout:setting min_mon_release = tentacle 2026-02-22T20:32:49.979 INFO:teuthology.orchestra.run.trial084.stdout:epoch 0 2026-02-22T20:32:49.979 INFO:teuthology.orchestra.run.trial084.stdout:fsid a4ea559c-fa00-404b-b21c-f34ee9bd0795 2026-02-22T20:32:49.979 INFO:teuthology.orchestra.run.trial084.stdout:last_changed 2026-02-22T20:32:49.977179+0000 2026-02-22T20:32:49.979 INFO:teuthology.orchestra.run.trial084.stdout:created 2026-02-22T20:32:49.977179+0000 2026-02-22T20:32:49.979 INFO:teuthology.orchestra.run.trial084.stdout:min_mon_release 20 (tentacle) 2026-02-22T20:32:49.979 INFO:teuthology.orchestra.run.trial084.stdout:election_strategy: 1 2026-02-22T20:32:49.979 INFO:teuthology.orchestra.run.trial084.stdout:0: [v2:10.20.193.84:3300/0,v1:10.20.193.84:6789/0] mon.a 2026-02-22T20:32:49.979 INFO:teuthology.orchestra.run.trial084.stdout:1: [v2:10.20.193.84:3301/0,v1:10.20.193.84:6790/0] mon.b 2026-02-22T20:32:49.980 INFO:teuthology.orchestra.run.trial084.stdout:2: [v2:10.20.193.84:3302/0,v1:10.20.193.84:6791/0] mon.c 2026-02-22T20:32:49.980 INFO:teuthology.orchestra.run.trial084.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2026-02-22T20:32:49.981 DEBUG:teuthology.orchestra.run.trial084:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-22T20:32:50.038 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a4ea559c-fa00-404b-b21c-f34ee9bd0795... 2026-02-22T20:32:50.039 DEBUG:teuthology.orchestra.run.trial084:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-22T20:32:50.122 INFO:teuthology.orchestra.run.trial084.stdout:[global] 2026-02-22T20:32:50.123 INFO:teuthology.orchestra.run.trial084.stdout: chdir = "" 2026-02-22T20:32:50.123 INFO:teuthology.orchestra.run.trial084.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-22T20:32:50.123 INFO:teuthology.orchestra.run.trial084.stdout: auth supported = cephx 2026-02-22T20:32:50.123 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.123 INFO:teuthology.orchestra.run.trial084.stdout: filestore xattr use omap = true 2026-02-22T20:32:50.123 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.123 INFO:teuthology.orchestra.run.trial084.stdout: mon clock drift allowed = 1.000 2026-02-22T20:32:50.123 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.123 INFO:teuthology.orchestra.run.trial084.stdout: osd crush chooseleaf type = 0 2026-02-22T20:32:50.123 INFO:teuthology.orchestra.run.trial084.stdout: auth debug = true 2026-02-22T20:32:50.124 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.124 INFO:teuthology.orchestra.run.trial084.stdout: ms die on old message = true 2026-02-22T20:32:50.124 INFO:teuthology.orchestra.run.trial084.stdout: ms die on bug = true 2026-02-22T20:32:50.124 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.124 INFO:teuthology.orchestra.run.trial084.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-22T20:32:50.124 INFO:teuthology.orchestra.run.trial084.stdout: mon pg warn max object skew = 0 2026-02-22T20:32:50.124 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.124 INFO:teuthology.orchestra.run.trial084.stdout: # disable pg_autoscaler by default for new pools 2026-02-22T20:32:50.124 INFO:teuthology.orchestra.run.trial084.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-22T20:32:50.124 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.124 INFO:teuthology.orchestra.run.trial084.stdout: osd pool default size = 2 2026-02-22T20:32:50.124 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.125 INFO:teuthology.orchestra.run.trial084.stdout: mon osd allow primary affinity = true 2026-02-22T20:32:50.125 INFO:teuthology.orchestra.run.trial084.stdout: mon osd allow pg remap = true 2026-02-22T20:32:50.125 INFO:teuthology.orchestra.run.trial084.stdout: mon warn on legacy crush tunables = false 2026-02-22T20:32:50.125 INFO:teuthology.orchestra.run.trial084.stdout: mon warn on crush straw calc version zero = false 2026-02-22T20:32:50.125 INFO:teuthology.orchestra.run.trial084.stdout: mon warn on no sortbitwise = false 2026-02-22T20:32:50.125 INFO:teuthology.orchestra.run.trial084.stdout: mon warn on osd down out interval zero = false 2026-02-22T20:32:50.125 INFO:teuthology.orchestra.run.trial084.stdout: mon warn on too few osds = false 2026-02-22T20:32:50.125 INFO:teuthology.orchestra.run.trial084.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-22T20:32:50.125 INFO:teuthology.orchestra.run.trial084.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-22T20:32:50.125 INFO:teuthology.orchestra.run.trial084.stdout: mon_allow_pool_size_one = true 2026-02-22T20:32:50.125 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.125 INFO:teuthology.orchestra.run.trial084.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-22T20:32:50.126 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.126 INFO:teuthology.orchestra.run.trial084.stdout: osd default data pool replay window = 5 2026-02-22T20:32:50.126 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.126 INFO:teuthology.orchestra.run.trial084.stdout: mon allow pool delete = true 2026-02-22T20:32:50.126 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.126 INFO:teuthology.orchestra.run.trial084.stdout: mon cluster log file level = debug 2026-02-22T20:32:50.126 INFO:teuthology.orchestra.run.trial084.stdout: debug asserts on shutdown = true 2026-02-22T20:32:50.126 INFO:teuthology.orchestra.run.trial084.stdout: mon health detail to clog = false 2026-02-22T20:32:50.126 INFO:teuthology.orchestra.run.trial084.stdout: mon host = "10.20.193.84,[v2:10.20.193.84:3301,v1:10.20.193.84:6790],[v2:10.20.193.84:3302,v1:10.20.193.84:6791]" 2026-02-22T20:32:50.126 INFO:teuthology.orchestra.run.trial084.stdout: mon election default strategy = 1 2026-02-22T20:32:50.126 INFO:teuthology.orchestra.run.trial084.stdout: ms type = async 2026-02-22T20:32:50.126 INFO:teuthology.orchestra.run.trial084.stdout: fsid = a4ea559c-fa00-404b-b21c-f34ee9bd0795 2026-02-22T20:32:50.127 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.127 INFO:teuthology.orchestra.run.trial084.stdout:[osd] 2026-02-22T20:32:50.127 INFO:teuthology.orchestra.run.trial084.stdout: osd journal size = 100 2026-02-22T20:32:50.127 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.127 INFO:teuthology.orchestra.run.trial084.stdout: osd scrub load threshold = 5.0 2026-02-22T20:32:50.127 INFO:teuthology.orchestra.run.trial084.stdout: osd scrub max interval = 600 2026-02-22T20:32:50.127 INFO:teuthology.orchestra.run.trial084.stdout: osd mclock profile = high_recovery_ops 2026-02-22T20:32:50.127 INFO:teuthology.orchestra.run.trial084.stdout: osd mclock skip benchmark = True 2026-02-22T20:32:50.127 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.127 INFO:teuthology.orchestra.run.trial084.stdout: osd recover clone overlap = true 2026-02-22T20:32:50.127 INFO:teuthology.orchestra.run.trial084.stdout: osd recovery max chunk = 1048576 2026-02-22T20:32:50.127 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.128 INFO:teuthology.orchestra.run.trial084.stdout: osd debug shutdown = true 2026-02-22T20:32:50.128 INFO:teuthology.orchestra.run.trial084.stdout: osd debug op order = true 2026-02-22T20:32:50.128 INFO:teuthology.orchestra.run.trial084.stdout: osd debug verify stray on activate = true 2026-02-22T20:32:50.128 INFO:teuthology.orchestra.run.trial084.stdout: osd debug trim objects = true 2026-02-22T20:32:50.128 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.128 INFO:teuthology.orchestra.run.trial084.stdout: osd open classes on start = true 2026-02-22T20:32:50.128 INFO:teuthology.orchestra.run.trial084.stdout: osd debug pg log writeout = true 2026-02-22T20:32:50.128 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.128 INFO:teuthology.orchestra.run.trial084.stdout: osd deep scrub update digest min age = 30 2026-02-22T20:32:50.128 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.128 INFO:teuthology.orchestra.run.trial084.stdout: osd map max advance = 10 2026-02-22T20:32:50.128 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: journal zero on create = true 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: filestore ondisk finisher threads = 3 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: filestore apply finisher threads = 3 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: bdev debug aio = true 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: osd debug misdirected ops = true 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: osd min pg log entries = 5 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: osd_fast_fail_on_connection_refused = False 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: bdev async discard = True 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: bdev enable discard = True 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: bluefs allocator = hybrid 2026-02-22T20:32:50.129 INFO:teuthology.orchestra.run.trial084.stdout: bluestore allocator = hybrid 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: bluestore block size = 96636764160 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: bluestore fsck on mount = True 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: bluestore onode segment size = 1024K 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: bluestore write v2 = True 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: bluestore zero block detection = True 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: debug bluefs = 20 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: debug bluestore = 20 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: debug ms = 1 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: debug osd = 20 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: debug rocksdb = 10 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: mon osd backfillfull_ratio = 0.85 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: mon osd full ratio = 0.9 2026-02-22T20:32:50.130 INFO:teuthology.orchestra.run.trial084.stdout: mon osd nearfull ratio = 0.8 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout: osd debug verify cached snaps = True 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout: osd debug verify missing on start = True 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout: osd failsafe full ratio = 0.95 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout: osd mclock override recovery settings = True 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout: osd memory target = 1610612736 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout: osd objectstore = bluestore 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout: osd op queue = debug_random 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout: osd op queue cut off = debug_random 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout:[mgr] 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout: debug ms = 1 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout: debug mgr = 20 2026-02-22T20:32:50.131 INFO:teuthology.orchestra.run.trial084.stdout: debug mon = 20 2026-02-22T20:32:50.132 INFO:teuthology.orchestra.run.trial084.stdout: debug auth = 20 2026-02-22T20:32:50.132 INFO:teuthology.orchestra.run.trial084.stdout: mon reweight min pgs per osd = 4 2026-02-22T20:32:50.132 INFO:teuthology.orchestra.run.trial084.stdout: mon reweight min bytes per osd = 10 2026-02-22T20:32:50.132 INFO:teuthology.orchestra.run.trial084.stdout: mgr/telemetry/nag = false 2026-02-22T20:32:50.132 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.132 INFO:teuthology.orchestra.run.trial084.stdout:[mon] 2026-02-22T20:32:50.132 INFO:teuthology.orchestra.run.trial084.stdout: debug ms = 1 2026-02-22T20:32:50.132 INFO:teuthology.orchestra.run.trial084.stdout: debug mon = 20 2026-02-22T20:32:50.132 INFO:teuthology.orchestra.run.trial084.stdout: debug paxos = 20 2026-02-22T20:32:50.132 INFO:teuthology.orchestra.run.trial084.stdout: debug auth = 20 2026-02-22T20:32:50.132 INFO:teuthology.orchestra.run.trial084.stdout: mon data avail warn = 5 2026-02-22T20:32:50.132 INFO:teuthology.orchestra.run.trial084.stdout: mon mgr mkfs grace = 240 2026-02-22T20:32:50.133 INFO:teuthology.orchestra.run.trial084.stdout: mon reweight min pgs per osd = 4 2026-02-22T20:32:50.133 INFO:teuthology.orchestra.run.trial084.stdout: mon osd reporter subtree level = osd 2026-02-22T20:32:50.133 INFO:teuthology.orchestra.run.trial084.stdout: mon osd prime pg temp = true 2026-02-22T20:32:50.133 INFO:teuthology.orchestra.run.trial084.stdout: mon reweight min bytes per osd = 10 2026-02-22T20:32:50.133 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.133 INFO:teuthology.orchestra.run.trial084.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-22T20:32:50.133 INFO:teuthology.orchestra.run.trial084.stdout: auth mon ticket ttl = 660 # 11m 2026-02-22T20:32:50.133 INFO:teuthology.orchestra.run.trial084.stdout: auth service ticket ttl = 240 # 4m 2026-02-22T20:32:50.133 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.133 INFO:teuthology.orchestra.run.trial084.stdout: # don't complain about insecure global_id in the test suite 2026-02-22T20:32:50.133 INFO:teuthology.orchestra.run.trial084.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-22T20:32:50.133 INFO:teuthology.orchestra.run.trial084.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-22T20:32:50.134 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.134 INFO:teuthology.orchestra.run.trial084.stdout: # 1m isn't quite enough 2026-02-22T20:32:50.134 INFO:teuthology.orchestra.run.trial084.stdout: mon_down_mkfs_grace = 2m 2026-02-22T20:32:50.134 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.134 INFO:teuthology.orchestra.run.trial084.stdout: mon_warn_on_filestore_osds = false 2026-02-22T20:32:50.134 INFO:teuthology.orchestra.run.trial084.stdout: mon scrub interval = 300 2026-02-22T20:32:50.134 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:32:50.134 INFO:teuthology.orchestra.run.trial084.stdout:[client] 2026-02-22T20:32:50.134 INFO:teuthology.orchestra.run.trial084.stdout: rgw cache enabled = true 2026-02-22T20:32:50.134 INFO:teuthology.orchestra.run.trial084.stdout: rgw enable ops log = true 2026-02-22T20:32:50.134 INFO:teuthology.orchestra.run.trial084.stdout: rgw enable usage log = true 2026-02-22T20:32:50.134 INFO:teuthology.orchestra.run.trial084.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-22T20:32:50.135 INFO:teuthology.orchestra.run.trial084.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-22T20:32:50.135 INFO:teuthology.orchestra.run.trial084.stdout:[mon.a] 2026-02-22T20:32:50.135 INFO:teuthology.orchestra.run.trial084.stdout:[mon.b] 2026-02-22T20:32:50.135 INFO:teuthology.orchestra.run.trial084.stdout:[mon.c] 2026-02-22T20:32:50.136 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-22T20:32:50.136 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-22T20:32:50.219 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-22T20:32:50.219 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:50.219 DEBUG:teuthology.orchestra.run.trial084:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-22T20:32:50.276 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:50.276 DEBUG:teuthology.orchestra.run.trial084:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-22T20:32:50.333 INFO:tasks.ceph:Sending monmap to node ubuntu@trial084.front.sepia.ceph.com 2026-02-22T20:32:50.333 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:50.333 DEBUG:teuthology.orchestra.run.trial084:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-22T20:32:50.333 DEBUG:teuthology.orchestra.run.trial084:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T20:32:50.409 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:50.409 DEBUG:teuthology.orchestra.run.trial084:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-22T20:32:50.466 INFO:tasks.ceph:Setting up mon nodes... 2026-02-22T20:32:50.466 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-22T20:32:50.466 DEBUG:teuthology.orchestra.run.trial084:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-22T20:32:50.552 INFO:teuthology.orchestra.run.trial084.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-22T20:32:50.555 INFO:tasks.ceph:Setting up mds nodes... 2026-02-22T20:32:50.555 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-22T20:32:50.555 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-22T20:32:50.556 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial084.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2026-02-22T20:32:50.556 DEBUG:teuthology.orchestra.run.trial084:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-22T20:32:50.584 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-22T20:32:50.584 INFO:tasks.ceph:role: osd.0 2026-02-22T20:32:50.584 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial084.front.sepia.ceph.com 2026-02-22T20:32:50.585 DEBUG:teuthology.orchestra.run.trial084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-22T20:32:50.651 INFO:teuthology.orchestra.run.trial084.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=24, agsize=7645866 blks 2026-02-22T20:32:50.651 INFO:teuthology.orchestra.run.trial084.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T20:32:50.651 INFO:teuthology.orchestra.run.trial084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=1 2026-02-22T20:32:50.651 INFO:teuthology.orchestra.run.trial084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=1 2026-02-22T20:32:50.651 INFO:teuthology.orchestra.run.trial084.stdout: = exchange=0 2026-02-22T20:32:50.652 INFO:teuthology.orchestra.run.trial084.stdout:data = bsize=4096 blocks=183500784, imaxpct=25 2026-02-22T20:32:50.652 INFO:teuthology.orchestra.run.trial084.stdout: = sunit=0 swidth=0 blks 2026-02-22T20:32:50.652 INFO:teuthology.orchestra.run.trial084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1, parent=0 2026-02-22T20:32:50.652 INFO:teuthology.orchestra.run.trial084.stdout:log =internal log bsize=4096 blocks=89599, version=2 2026-02-22T20:32:50.652 INFO:teuthology.orchestra.run.trial084.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T20:32:50.652 INFO:teuthology.orchestra.run.trial084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T20:32:51.246 INFO:teuthology.orchestra.run.trial084.stdout:Discarding blocks...Done. 2026-02-22T20:32:51.247 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial084.front.sepia.ceph.com -o noatime 2026-02-22T20:32:51.247 DEBUG:teuthology.orchestra.run.trial084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-22T20:32:51.340 DEBUG:teuthology.orchestra.run.trial084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-22T20:32:51.369 DEBUG:teuthology.orchestra.run.trial084:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-22T20:32:51.437 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-22T20:32:51.437 INFO:tasks.ceph:role: osd.1 2026-02-22T20:32:51.437 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial084.front.sepia.ceph.com 2026-02-22T20:32:51.437 DEBUG:teuthology.orchestra.run.trial084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-22T20:32:51.504 INFO:teuthology.orchestra.run.trial084.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=24, agsize=7645866 blks 2026-02-22T20:32:51.504 INFO:teuthology.orchestra.run.trial084.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T20:32:51.504 INFO:teuthology.orchestra.run.trial084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=1 2026-02-22T20:32:51.504 INFO:teuthology.orchestra.run.trial084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=1 2026-02-22T20:32:51.504 INFO:teuthology.orchestra.run.trial084.stdout: = exchange=0 2026-02-22T20:32:51.504 INFO:teuthology.orchestra.run.trial084.stdout:data = bsize=4096 blocks=183500784, imaxpct=25 2026-02-22T20:32:51.504 INFO:teuthology.orchestra.run.trial084.stdout: = sunit=0 swidth=0 blks 2026-02-22T20:32:51.504 INFO:teuthology.orchestra.run.trial084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1, parent=0 2026-02-22T20:32:51.504 INFO:teuthology.orchestra.run.trial084.stdout:log =internal log bsize=4096 blocks=89599, version=2 2026-02-22T20:32:51.504 INFO:teuthology.orchestra.run.trial084.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T20:32:51.504 INFO:teuthology.orchestra.run.trial084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T20:32:52.094 INFO:teuthology.orchestra.run.trial084.stdout:Discarding blocks...Done. 2026-02-22T20:32:52.096 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial084.front.sepia.ceph.com -o noatime 2026-02-22T20:32:52.096 DEBUG:teuthology.orchestra.run.trial084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-22T20:32:52.187 DEBUG:teuthology.orchestra.run.trial084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-22T20:32:52.216 DEBUG:teuthology.orchestra.run.trial084:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-22T20:32:52.284 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-22T20:32:52.285 INFO:tasks.ceph:role: osd.2 2026-02-22T20:32:52.285 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial084.front.sepia.ceph.com 2026-02-22T20:32:52.285 DEBUG:teuthology.orchestra.run.trial084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-22T20:32:52.352 INFO:teuthology.orchestra.run.trial084.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=24, agsize=7645866 blks 2026-02-22T20:32:52.352 INFO:teuthology.orchestra.run.trial084.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T20:32:52.352 INFO:teuthology.orchestra.run.trial084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=1 2026-02-22T20:32:52.352 INFO:teuthology.orchestra.run.trial084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=1 2026-02-22T20:32:52.352 INFO:teuthology.orchestra.run.trial084.stdout: = exchange=0 2026-02-22T20:32:52.352 INFO:teuthology.orchestra.run.trial084.stdout:data = bsize=4096 blocks=183500784, imaxpct=25 2026-02-22T20:32:52.352 INFO:teuthology.orchestra.run.trial084.stdout: = sunit=0 swidth=0 blks 2026-02-22T20:32:52.352 INFO:teuthology.orchestra.run.trial084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1, parent=0 2026-02-22T20:32:52.352 INFO:teuthology.orchestra.run.trial084.stdout:log =internal log bsize=4096 blocks=89599, version=2 2026-02-22T20:32:52.352 INFO:teuthology.orchestra.run.trial084.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T20:32:52.352 INFO:teuthology.orchestra.run.trial084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T20:32:52.934 INFO:teuthology.orchestra.run.trial084.stdout:Discarding blocks...Done. 2026-02-22T20:32:52.936 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial084.front.sepia.ceph.com -o noatime 2026-02-22T20:32:52.936 DEBUG:teuthology.orchestra.run.trial084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-22T20:32:53.027 DEBUG:teuthology.orchestra.run.trial084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-22T20:32:53.055 DEBUG:teuthology.orchestra.run.trial084:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-22T20:32:53.123 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-22T20:32:53.123 INFO:tasks.ceph:role: osd.3 2026-02-22T20:32:53.123 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial084.front.sepia.ceph.com 2026-02-22T20:32:53.123 DEBUG:teuthology.orchestra.run.trial084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-22T20:32:53.190 INFO:teuthology.orchestra.run.trial084.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=24, agsize=7645866 blks 2026-02-22T20:32:53.191 INFO:teuthology.orchestra.run.trial084.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T20:32:53.191 INFO:teuthology.orchestra.run.trial084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=1 2026-02-22T20:32:53.191 INFO:teuthology.orchestra.run.trial084.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=1 2026-02-22T20:32:53.191 INFO:teuthology.orchestra.run.trial084.stdout: = exchange=0 2026-02-22T20:32:53.191 INFO:teuthology.orchestra.run.trial084.stdout:data = bsize=4096 blocks=183500784, imaxpct=25 2026-02-22T20:32:53.191 INFO:teuthology.orchestra.run.trial084.stdout: = sunit=0 swidth=0 blks 2026-02-22T20:32:53.191 INFO:teuthology.orchestra.run.trial084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1, parent=0 2026-02-22T20:32:53.191 INFO:teuthology.orchestra.run.trial084.stdout:log =internal log bsize=4096 blocks=89599, version=2 2026-02-22T20:32:53.191 INFO:teuthology.orchestra.run.trial084.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T20:32:53.191 INFO:teuthology.orchestra.run.trial084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T20:32:53.773 INFO:teuthology.orchestra.run.trial084.stdout:Discarding blocks...Done. 2026-02-22T20:32:53.774 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial084.front.sepia.ceph.com -o noatime 2026-02-22T20:32:53.775 DEBUG:teuthology.orchestra.run.trial084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2026-02-22T20:32:53.865 DEBUG:teuthology.orchestra.run.trial084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-22T20:32:53.895 DEBUG:teuthology.orchestra.run.trial084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T20:32:53.975 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:53.973+0000 7f0b9a77f900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-22T20:32:53.976 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:53.974+0000 7f0b9a77f900 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-22T20:32:53.976 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:53.974+0000 7f0b9a77f900 -1 bdev(0x55c8e29df800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-22T20:32:53.976 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:53.974+0000 7f0b9a77f900 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-22T20:32:54.514 DEBUG:teuthology.orchestra.run.trial084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-22T20:32:54.583 DEBUG:teuthology.orchestra.run.trial084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T20:32:54.662 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:54.659+0000 7fcf89233900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-22T20:32:54.662 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:54.659+0000 7fcf89233900 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-22T20:32:54.662 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:54.660+0000 7fcf89233900 -1 bdev(0x5630964d5800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-22T20:32:54.662 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:54.660+0000 7fcf89233900 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-22T20:32:55.180 DEBUG:teuthology.orchestra.run.trial084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-22T20:32:55.250 DEBUG:teuthology.orchestra.run.trial084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T20:32:55.330 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:55.328+0000 7fa9b9a55900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-22T20:32:55.330 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:55.328+0000 7fa9b9a55900 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-22T20:32:55.330 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:55.328+0000 7fa9b9a55900 -1 bdev(0x55b61aa2b800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-22T20:32:55.330 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:55.328+0000 7fa9b9a55900 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-22T20:32:55.814 DEBUG:teuthology.orchestra.run.trial084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-22T20:32:55.883 DEBUG:teuthology.orchestra.run.trial084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T20:32:55.964 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:55.961+0000 7ff821e6a900 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2026-02-22T20:32:55.964 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:55.961+0000 7ff821e6a900 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2026-02-22T20:32:55.964 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:55.962+0000 7ff821e6a900 -1 bdev(0x55fecc27f800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2026-02-22T20:32:55.964 INFO:teuthology.orchestra.run.trial084.stderr:2026-02-22T20:32:55.962+0000 7ff821e6a900 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2026-02-22T20:32:56.478 DEBUG:teuthology.orchestra.run.trial084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-22T20:32:56.547 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-22T20:32:56.548 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:56.548 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-22T20:32:56.615 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:56.615 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-22T20:32:56.680 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:56.681 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-22T20:32:56.747 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:56.747 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-22T20:32:56.817 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:32:56.817 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-22T20:32:56.884 INFO:tasks.ceph:Adding keys to all mons... 2026-02-22T20:32:56.884 DEBUG:teuthology.orchestra.run.trial084:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-22T20:32:56.949 INFO:teuthology.orchestra.run.trial084.stdout:[mgr.x] 2026-02-22T20:32:56.949 INFO:teuthology.orchestra.run.trial084.stdout: key = AQDyZ5tp4MTfIBAA+oLaNM0sWgAx+YMNfjdWFQ== 2026-02-22T20:32:56.949 INFO:teuthology.orchestra.run.trial084.stdout:[osd.0] 2026-02-22T20:32:56.949 INFO:teuthology.orchestra.run.trial084.stdout: key = AQD1Z5tpjR8UOhAAaBd7Dhm0jIRoOae8U6giWg== 2026-02-22T20:32:56.949 INFO:teuthology.orchestra.run.trial084.stdout:[osd.1] 2026-02-22T20:32:56.949 INFO:teuthology.orchestra.run.trial084.stdout: key = AQD2Z5tpiv5bJxAAlX9/Re+NcTag8W0aMGvqNg== 2026-02-22T20:32:56.949 INFO:teuthology.orchestra.run.trial084.stdout:[osd.2] 2026-02-22T20:32:56.950 INFO:teuthology.orchestra.run.trial084.stdout: key = AQD3Z5tpceqZExAA3kLAcIsaKMMF3m196qsoeQ== 2026-02-22T20:32:56.950 INFO:teuthology.orchestra.run.trial084.stdout:[osd.3] 2026-02-22T20:32:56.950 INFO:teuthology.orchestra.run.trial084.stdout: key = AQD3Z5tpV21cORAAbxzZm6v2m7Ov2WGT8QceRQ== 2026-02-22T20:32:56.951 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-22T20:32:57.032 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T20:32:57.112 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T20:32:57.192 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T20:32:57.273 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T20:32:57.353 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-22T20:32:57.353 DEBUG:teuthology.orchestra.run.trial084:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-22T20:32:57.419 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-22T20:32:57.505 DEBUG:teuthology.orchestra.run.trial084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-22T20:32:57.533 DEBUG:teuthology.orchestra.run.trial084:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2026-02-22T20:32:57.600 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-22T20:32:57.687 DEBUG:teuthology.orchestra.run.trial084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2026-02-22T20:32:57.714 DEBUG:teuthology.orchestra.run.trial084:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2026-02-22T20:32:57.781 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-22T20:32:57.866 DEBUG:teuthology.orchestra.run.trial084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2026-02-22T20:32:57.895 DEBUG:teuthology.orchestra.run.trial084:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-22T20:32:57.952 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-22T20:32:57.952 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-22T20:32:57.953 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-22T20:32:57.994 INFO:tasks.ceph.mon.a:Started 2026-02-22T20:32:57.994 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-22T20:32:57.995 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-22T20:32:57.997 INFO:tasks.ceph.mon.b:Started 2026-02-22T20:32:57.997 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-22T20:32:57.997 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-22T20:32:57.999 INFO:tasks.ceph.mon.c:Started 2026-02-22T20:32:58.000 DEBUG:teuthology.orchestra.run.trial084:> sudo ceph config set mgr mgr_pool false --force 2026-02-22T20:33:04.128 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-22T20:33:04.128 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-22T20:33:04.128 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-22T20:33:04.129 INFO:tasks.ceph.mgr.x:Started 2026-02-22T20:33:04.129 DEBUG:tasks.ceph:set 0 configs 2026-02-22T20:33:04.129 DEBUG:teuthology.orchestra.run.trial084:> sudo ceph --cluster ceph config dump 2026-02-22T20:33:04.192 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.189+0000 7f5820734140 -1 mgr[py] Loading module orchestrator in main interpreter 2026-02-22T20:33:04.240 INFO:teuthology.orchestra.run.trial084.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-22T20:33:04.240 INFO:teuthology.orchestra.run.trial084.stdout:mgr dev mgr_pool false * 2026-02-22T20:33:04.251 INFO:tasks.ceph:Setting crush tunables to default 2026-02-22T20:33:04.251 DEBUG:teuthology.orchestra.run.trial084:> sudo ceph --cluster ceph osd crush tunables default 2026-02-22T20:33:04.288 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.285+0000 7f5820734140 -1 mgr[py] Loading module dashboard in main interpreter 2026-02-22T20:33:04.361 INFO:teuthology.orchestra.run.trial084.stderr:adjusted tunables profile to default 2026-02-22T20:33:04.373 INFO:tasks.ceph:check_enable_crimson: False 2026-02-22T20:33:04.373 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-22T20:33:04.374 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:33:04.374 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-22T20:33:04.438 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:33:04.438 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-22T20:33:04.497 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.494+0000 7f5820734140 -1 mgr[py] Loading module alerts in main interpreter 2026-02-22T20:33:04.499 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.496+0000 7f5820734140 -1 mgr[py] Loading module smb in main interpreter 2026-02-22T20:33:04.504 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:33:04.504 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-22T20:33:04.524 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.521+0000 7f5820734140 -1 mgr[py] Loading module snap_schedule in main interpreter 2026-02-22T20:33:04.530 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.528+0000 7f5820734140 -1 mgr[py] Loading module stats in main interpreter 2026-02-22T20:33:04.533 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.531+0000 7f5820734140 -1 mgr[py] Loading module devicehealth in main interpreter 2026-02-22T20:33:04.536 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.534+0000 7f5820734140 -1 mgr[py] Loading module iostat in main interpreter 2026-02-22T20:33:04.537 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.534+0000 7f5820734140 -1 mgr[py] Loading module influx in main interpreter 2026-02-22T20:33:04.539 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.536+0000 7f5820734140 -1 mgr[py] Loading module telegraf in main interpreter 2026-02-22T20:33:04.541 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.539+0000 7f5820734140 -1 mgr[py] Loading module volumes in main interpreter 2026-02-22T20:33:04.549 DEBUG:teuthology.orchestra.run.trial084:> set -ex 2026-02-22T20:33:04.550 DEBUG:teuthology.orchestra.run.trial084:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-22T20:33:04.570 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.567+0000 7f5820734140 -1 mgr[py] Loading module rbd_support in main interpreter 2026-02-22T20:33:04.582 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.580+0000 7f5820734140 -1 mgr[py] Loading module mirroring in main interpreter 2026-02-22T20:33:04.592 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.589+0000 7f5820734140 -1 mgr[py] Loading module rgw in main interpreter 2026-02-22T20:33:04.595 DEBUG:teuthology.orchestra.run.trial084:> sudo ceph --cluster ceph osd new ca475199-167f-4aba-960a-1e3782214a73 0 2026-02-22T20:33:04.596 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.593+0000 7f5820734140 -1 mgr[py] Loading module pg_autoscaler in main interpreter 2026-02-22T20:33:04.599 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.596+0000 7f5820734140 -1 mgr[py] Loading module mds_autoscaler in main interpreter 2026-02-22T20:33:04.600 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.597+0000 7f5820734140 -1 mgr[py] Loading module localpool in main interpreter 2026-02-22T20:33:04.601 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.598+0000 7f5820734140 -1 mgr[py] Loading module crash in main interpreter 2026-02-22T20:33:04.603 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.600+0000 7f5820734140 -1 mgr[py] Loading module prometheus in main interpreter 2026-02-22T20:33:04.610 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.608+0000 7f5820734140 -1 mgr[py] Loading module insights in main interpreter 2026-02-22T20:33:04.612 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.610+0000 7f5820734140 -1 mgr[py] Loading module test_orchestrator in main interpreter 2026-02-22T20:33:04.614 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.611+0000 7f5820734140 -1 mgr[py] Loading module selftest in main interpreter 2026-02-22T20:33:04.617 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.614+0000 7f5820734140 -1 mgr[py] Loading module cephadm in main interpreter 2026-02-22T20:33:04.710 INFO:teuthology.orchestra.run.trial084.stdout:0 2026-02-22T20:33:04.722 DEBUG:teuthology.orchestra.run.trial084:> sudo ceph --cluster ceph osd new 32df476a-2b69-493e-a636-1ba8bf5268a2 1 2026-02-22T20:33:04.785 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.782+0000 7f5820734140 -1 mgr[py] Loading module nfs in main interpreter 2026-02-22T20:33:04.794 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.791+0000 7f5820734140 -1 mgr[py] Loading module telemetry in main interpreter 2026-02-22T20:33:04.801 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.798+0000 7f5820734140 -1 mgr[py] Loading module progress in main interpreter 2026-02-22T20:33:04.803 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.801+0000 7f5820734140 -1 mgr[py] Loading module diskprediction_local in main interpreter 2026-02-22T20:33:04.844 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.841+0000 7f5820734140 -1 mgr[py] Loading module rook in main interpreter 2026-02-22T20:33:04.880 INFO:teuthology.orchestra.run.trial084.stdout:1 2026-02-22T20:33:04.892 DEBUG:teuthology.orchestra.run.trial084:> sudo ceph --cluster ceph osd new 196c3661-b3be-46a3-9f14-7908d9003447 2 2026-02-22T20:33:04.971 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.967+0000 7f5820734140 -1 mgr[py] Loading module balancer in main interpreter 2026-02-22T20:33:04.977 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.974+0000 7f5820734140 -1 mgr[py] Loading module osd_perf_query in main interpreter 2026-02-22T20:33:04.978 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.975+0000 7f5820734140 -1 mgr[py] Loading module status in main interpreter 2026-02-22T20:33:04.979 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:04.977+0000 7f5820734140 -1 mgr[py] Loading module osd_support in main interpreter 2026-02-22T20:33:05.050 INFO:teuthology.orchestra.run.trial084.stdout:2 2026-02-22T20:33:05.061 DEBUG:teuthology.orchestra.run.trial084:> sudo ceph --cluster ceph osd new bdc6271f-361f-4362-b050-080c613ab56c 3 2026-02-22T20:33:05.216 INFO:teuthology.orchestra.run.trial084.stdout:3 2026-02-22T20:33:05.228 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-22T20:33:05.228 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2026-02-22T20:33:05.230 INFO:tasks.ceph.osd.0:Started 2026-02-22T20:33:05.230 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-22T20:33:05.231 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2026-02-22T20:33:05.233 INFO:tasks.ceph.osd.1:Started 2026-02-22T20:33:05.233 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-22T20:33:05.233 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-22T20:33:05.235 INFO:tasks.ceph.osd.2:Started 2026-02-22T20:33:05.236 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-22T20:33:05.236 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2026-02-22T20:33:05.238 INFO:tasks.ceph.osd.3:Started 2026-02-22T20:33:05.238 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T20:33:05.351 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:05.351 INFO:teuthology.orchestra.run.trial084.stdout:{"epoch":6,"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","created":"2026-02-22T20:33:03.270442+0000","modified":"2026-02-22T20:33:05.214562+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T20:33:05.214562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ca475199-167f-4aba-960a-1e3782214a73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"32df476a-2b69-493e-a636-1ba8bf5268a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"196c3661-b3be-46a3-9f14-7908d9003447","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bdc6271f-361f-4362-b050-080c613ab56c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T20:33:05.362 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-22T20:33:05.362 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-22T20:33:05.505 INFO:tasks.ceph.osd.0.trial084.stderr:2026-02-22T20:33:05.502+0000 7fac0d51d900 -1 Falling back to public interface 2026-02-22T20:33:05.514 INFO:tasks.ceph.osd.3.trial084.stderr:2026-02-22T20:33:05.511+0000 7fa13cfc8900 -1 Falling back to public interface 2026-02-22T20:33:05.515 INFO:tasks.ceph.osd.1.trial084.stderr:2026-02-22T20:33:05.511+0000 7fd4f69c6900 -1 Falling back to public interface 2026-02-22T20:33:05.520 INFO:tasks.ceph.osd.2.trial084.stderr:2026-02-22T20:33:05.517+0000 7f4590100900 -1 Falling back to public interface 2026-02-22T20:33:05.765 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T20:33:05.869 INFO:teuthology.misc.health.trial084.stdout: 2026-02-22T20:33:05.869 INFO:teuthology.misc.health.trial084.stdout:{"epoch":6,"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","created":"2026-02-22T20:33:03.270442+0000","modified":"2026-02-22T20:33:05.214562+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T20:33:05.214562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ca475199-167f-4aba-960a-1e3782214a73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"32df476a-2b69-493e-a636-1ba8bf5268a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"196c3661-b3be-46a3-9f14-7908d9003447","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bdc6271f-361f-4362-b050-080c613ab56c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T20:33:05.881 DEBUG:teuthology.misc:0 of 4 OSDs are up 2026-02-22T20:33:05.990 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:05.987+0000 7f574acfd6c0 -1 mgr.server handle_report got status from non-daemon mon.b 2026-02-22T20:33:05.990 INFO:tasks.ceph.mgr.x.trial084.stderr:2026-02-22T20:33:05.987+0000 7f574acfd6c0 -1 mgr.server handle_report got status from non-daemon mon.c 2026-02-22T20:33:06.147 INFO:tasks.ceph.osd.0.trial084.stderr:2026-02-22T20:33:06.144+0000 7fac0d51d900 -1 osd.0 0 log_to_monitors true 2026-02-22T20:33:06.150 INFO:tasks.ceph.osd.1.trial084.stderr:2026-02-22T20:33:06.148+0000 7fd4f69c6900 -1 osd.1 0 log_to_monitors true 2026-02-22T20:33:06.151 INFO:tasks.ceph.osd.3.trial084.stderr:2026-02-22T20:33:06.148+0000 7fa13cfc8900 -1 osd.3 0 log_to_monitors true 2026-02-22T20:33:06.170 INFO:tasks.ceph.osd.2.trial084.stderr:2026-02-22T20:33:06.167+0000 7f4590100900 -1 osd.2 0 log_to_monitors true 2026-02-22T20:33:08.001 INFO:tasks.ceph.osd.1.trial084.stderr:2026-02-22T20:33:07.998+0000 7fd4f21516c0 -1 osd.1 0 waiting for initial osdmap 2026-02-22T20:33:08.001 INFO:tasks.ceph.osd.2.trial084.stderr:2026-02-22T20:33:07.998+0000 7f458b88b6c0 -1 osd.2 0 waiting for initial osdmap 2026-02-22T20:33:08.001 INFO:tasks.ceph.osd.3.trial084.stderr:2026-02-22T20:33:07.998+0000 7fa1387536c0 -1 osd.3 0 waiting for initial osdmap 2026-02-22T20:33:08.002 INFO:tasks.ceph.osd.0.trial084.stderr:2026-02-22T20:33:07.999+0000 7fac08ca86c0 -1 osd.0 0 waiting for initial osdmap 2026-02-22T20:33:08.002 INFO:tasks.ceph.osd.1.trial084.stderr:2026-02-22T20:33:08.000+0000 7fd4e9f586c0 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T20:33:08.003 INFO:tasks.ceph.osd.2.trial084.stderr:2026-02-22T20:33:08.000+0000 7f458769a6c0 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T20:33:08.003 INFO:tasks.ceph.osd.3.trial084.stderr:2026-02-22T20:33:08.001+0000 7fa13055a6c0 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T20:33:08.004 INFO:tasks.ceph.osd.0.trial084.stderr:2026-02-22T20:33:08.001+0000 7fac04ab76c0 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T20:33:12.284 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T20:33:12.454 INFO:teuthology.misc.health.trial084.stdout: 2026-02-22T20:33:12.454 INFO:teuthology.misc.health.trial084.stdout:{"epoch":9,"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","created":"2026-02-22T20:33:03.270442+0000","modified":"2026-02-22T20:33:08.988844+0000","last_up_change":"2026-02-22T20:33:08.988844+0000","last_in_change":"2026-02-22T20:33:05.214562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ca475199-167f-4aba-960a-1e3782214a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6802","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6803","nonce":1867649424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6804","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6805","nonce":1867649424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6808","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6809","nonce":1867649424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6806","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6807","nonce":1867649424}]},"public_addr":"10.20.193.84:6803/1867649424","cluster_addr":"10.20.193.84:6805/1867649424","heartbeat_back_addr":"10.20.193.84:6809/1867649424","heartbeat_front_addr":"10.20.193.84:6807/1867649424","state":["exists","up"]},{"osd":1,"uuid":"32df476a-2b69-493e-a636-1ba8bf5268a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6811","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6813","nonce":2769530292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6815","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6818","nonce":2769530292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6824","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6825","nonce":2769530292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6820","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6822","nonce":2769530292}]},"public_addr":"10.20.193.84:6813/2769530292","cluster_addr":"10.20.193.84:6818/2769530292","heartbeat_back_addr":"10.20.193.84:6825/2769530292","heartbeat_front_addr":"10.20.193.84:6822/2769530292","state":["exists","up"]},{"osd":2,"uuid":"196c3661-b3be-46a3-9f14-7908d9003447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6826","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6827","nonce":2089419628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6828","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6829","nonce":2089419628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6832","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6833","nonce":2089419628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6830","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6831","nonce":2089419628}]},"public_addr":"10.20.193.84:6827/2089419628","cluster_addr":"10.20.193.84:6829/2089419628","heartbeat_back_addr":"10.20.193.84:6833/2089419628","heartbeat_front_addr":"10.20.193.84:6831/2089419628","state":["exists","up"]},{"osd":3,"uuid":"bdc6271f-361f-4362-b050-080c613ab56c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6810","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6812","nonce":2227945027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6814","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6816","nonce":2227945027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6821","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6823","nonce":2227945027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6817","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6819","nonce":2227945027}]},"public_addr":"10.20.193.84:6812/2227945027","cluster_addr":"10.20.193.84:6816/2227945027","heartbeat_back_addr":"10.20.193.84:6823/2227945027","heartbeat_front_addr":"10.20.193.84:6819/2227945027","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T20:33:12.464 DEBUG:teuthology.misc:4 of 4 OSDs are up 2026-02-22T20:33:12.464 INFO:tasks.ceph:Creating RBD pool 2026-02-22T20:33:12.464 DEBUG:teuthology.orchestra.run.trial084:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-22T20:33:13.003 INFO:teuthology.orchestra.run.trial084.stderr:pool 'rbd' created 2026-02-22T20:33:13.018 DEBUG:teuthology.orchestra.run.trial084:> rbd --cluster ceph pool init rbd 2026-02-22T20:33:16.019 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-22T20:33:16.020 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-22T20:33:16.021 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-22T20:33:16.203 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:16.215 INFO:teuthology.orchestra.run.trial084.stdout:[{"version":2,"timestamp":"2026-02-22T20:33:04.113643+0000","name":"","changes":[{"name":"mgr/mgr_pool","new_value":"false"}]}] 2026-02-22T20:33:16.215 INFO:tasks.ceph_manager:config epoch is 2 2026-02-22T20:33:16.215 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-22T20:33:16.215 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-22T20:33:16.216 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-22T20:33:16.419 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:16.430 INFO:teuthology.orchestra.run.trial084.stdout:{"epoch":4,"flags":0,"active_gid":4110,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6800","nonce":3717128314},{"type":"v1","addr":"10.20.193.84:6801","nonce":3717128314}]},"active_addr":"10.20.193.84:6801/3717128314","active_change":"2026-02-22T20:33:04.979920+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self certificates generated and signed by cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.5","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:ceph20-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:ceph20-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"how frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.84:0","nonce":1832782282}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.84:0","nonce":4014732529}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.84:0","nonce":3046515550}]}]} 2026-02-22T20:33:16.432 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-22T20:33:16.432 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-22T20:33:16.433 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T20:33:16.610 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:16.611 INFO:teuthology.orchestra.run.trial084.stdout:{"epoch":13,"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","created":"2026-02-22T20:33:03.270442+0000","modified":"2026-02-22T20:33:16.007814+0000","last_up_change":"2026-02-22T20:33:08.988844+0000","last_in_change":"2026-02-22T20:33:05.214562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-22T20:33:12.644417+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca475199-167f-4aba-960a-1e3782214a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6802","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6803","nonce":1867649424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6804","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6805","nonce":1867649424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6808","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6809","nonce":1867649424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6806","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6807","nonce":1867649424}]},"public_addr":"10.20.193.84:6803/1867649424","cluster_addr":"10.20.193.84:6805/1867649424","heartbeat_back_addr":"10.20.193.84:6809/1867649424","heartbeat_front_addr":"10.20.193.84:6807/1867649424","state":["exists","up"]},{"osd":1,"uuid":"32df476a-2b69-493e-a636-1ba8bf5268a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6811","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6813","nonce":2769530292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6815","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6818","nonce":2769530292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6824","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6825","nonce":2769530292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6820","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6822","nonce":2769530292}]},"public_addr":"10.20.193.84:6813/2769530292","cluster_addr":"10.20.193.84:6818/2769530292","heartbeat_back_addr":"10.20.193.84:6825/2769530292","heartbeat_front_addr":"10.20.193.84:6822/2769530292","state":["exists","up"]},{"osd":2,"uuid":"196c3661-b3be-46a3-9f14-7908d9003447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6826","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6827","nonce":2089419628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6828","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6829","nonce":2089419628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6832","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6833","nonce":2089419628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6830","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6831","nonce":2089419628}]},"public_addr":"10.20.193.84:6827/2089419628","cluster_addr":"10.20.193.84:6829/2089419628","heartbeat_back_addr":"10.20.193.84:6833/2089419628","heartbeat_front_addr":"10.20.193.84:6831/2089419628","state":["exists","up"]},{"osd":3,"uuid":"bdc6271f-361f-4362-b050-080c613ab56c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6810","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6812","nonce":2227945027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6814","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6816","nonce":2227945027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6821","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6823","nonce":2227945027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6817","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6819","nonce":2227945027}]},"public_addr":"10.20.193.84:6812/2227945027","cluster_addr":"10.20.193.84:6816/2227945027","heartbeat_back_addr":"10.20.193.84:6823/2227945027","heartbeat_front_addr":"10.20.193.84:6819/2227945027","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T20:33:16.623 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-22T20:33:16.623 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T20:33:16.800 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:16.800 INFO:teuthology.orchestra.run.trial084.stdout:{"epoch":13,"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","created":"2026-02-22T20:33:03.270442+0000","modified":"2026-02-22T20:33:16.007814+0000","last_up_change":"2026-02-22T20:33:08.988844+0000","last_in_change":"2026-02-22T20:33:05.214562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-22T20:33:12.644417+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca475199-167f-4aba-960a-1e3782214a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6802","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6803","nonce":1867649424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6804","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6805","nonce":1867649424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6808","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6809","nonce":1867649424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6806","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6807","nonce":1867649424}]},"public_addr":"10.20.193.84:6803/1867649424","cluster_addr":"10.20.193.84:6805/1867649424","heartbeat_back_addr":"10.20.193.84:6809/1867649424","heartbeat_front_addr":"10.20.193.84:6807/1867649424","state":["exists","up"]},{"osd":1,"uuid":"32df476a-2b69-493e-a636-1ba8bf5268a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6811","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6813","nonce":2769530292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6815","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6818","nonce":2769530292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6824","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6825","nonce":2769530292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6820","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6822","nonce":2769530292}]},"public_addr":"10.20.193.84:6813/2769530292","cluster_addr":"10.20.193.84:6818/2769530292","heartbeat_back_addr":"10.20.193.84:6825/2769530292","heartbeat_front_addr":"10.20.193.84:6822/2769530292","state":["exists","up"]},{"osd":2,"uuid":"196c3661-b3be-46a3-9f14-7908d9003447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6826","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6827","nonce":2089419628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6828","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6829","nonce":2089419628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6832","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6833","nonce":2089419628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6830","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6831","nonce":2089419628}]},"public_addr":"10.20.193.84:6827/2089419628","cluster_addr":"10.20.193.84:6829/2089419628","heartbeat_back_addr":"10.20.193.84:6833/2089419628","heartbeat_front_addr":"10.20.193.84:6831/2089419628","state":["exists","up"]},{"osd":3,"uuid":"bdc6271f-361f-4362-b050-080c613ab56c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6810","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6812","nonce":2227945027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6814","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6816","nonce":2227945027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6821","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6823","nonce":2227945027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6817","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6819","nonce":2227945027}]},"public_addr":"10.20.193.84:6812/2227945027","cluster_addr":"10.20.193.84:6816/2227945027","heartbeat_back_addr":"10.20.193.84:6823/2227945027","heartbeat_front_addr":"10.20.193.84:6819/2227945027","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T20:33:16.811 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-22T20:33:16.811 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-22T20:33:16.812 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-22T20:33:16.812 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-22T20:33:16.920 INFO:teuthology.orchestra.run.trial084.stdout:38654705669 2026-02-22T20:33:16.921 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-22T20:33:16.922 INFO:teuthology.orchestra.run.trial084.stdout:38654705667 2026-02-22T20:33:16.923 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T20:33:16.924 INFO:teuthology.orchestra.run.trial084.stdout:38654705668 2026-02-22T20:33:16.924 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T20:33:16.925 INFO:teuthology.orchestra.run.trial084.stdout:38654705668 2026-02-22T20:33:16.926 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T20:33:17.145 INFO:teuthology.orchestra.run.trial084.stdout:38654705669 2026-02-22T20:33:17.145 INFO:teuthology.orchestra.run.trial084.stdout:38654705668 2026-02-22T20:33:17.146 INFO:teuthology.orchestra.run.trial084.stdout:38654705668 2026-02-22T20:33:17.149 INFO:teuthology.orchestra.run.trial084.stdout:38654705667 2026-02-22T20:33:17.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.2 2026-02-22T20:33:17.161 DEBUG:teuthology.parallel:result is None 2026-02-22T20:33:17.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.3 2026-02-22T20:33:17.162 DEBUG:teuthology.parallel:result is None 2026-02-22T20:33:17.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.0 2026-02-22T20:33:17.164 DEBUG:teuthology.parallel:result is None 2026-02-22T20:33:17.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.1 2026-02-22T20:33:17.165 DEBUG:teuthology.parallel:result is None 2026-02-22T20:33:17.165 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-22T20:33:17.165 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:17.385 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:17.385 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:17.398 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2026-02-22T20:33:16.984141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":108280,"kb_used_data":440,"kb_used_omap":25,"kb_used_meta":107750,"kb_avail":377379080,"statfs":{"total":386547056640,"available":386436177920,"internally_reserved":0,"allocated":450560,"data_stored":118641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25984,"internal_metadata":110336640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.977955"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.012268+0000","last_change":"2026-02-22T20:33:16.012562+0000","last_active":"2026-02-22T20:33:16.012268+0000","last_peered":"2026-02-22T20:33:16.012268+0000","last_clean":"2026-02-22T20:33:16.012268+0000","last_became_active":"2026-02-22T20:33:14.005292+0000","last_became_peered":"2026-02-22T20:33:14.005292+0000","last_unstale":"2026-02-22T20:33:16.012268+0000","last_undegraded":"2026-02-22T20:33:16.012268+0000","last_fullsized":"2026-02-22T20:33:16.012268+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:19:42.247294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045825,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.012239+0000","last_change":"2026-02-22T20:33:16.012620+0000","last_active":"2026-02-22T20:33:16.012239+0000","last_peered":"2026-02-22T20:33:16.012239+0000","last_clean":"2026-02-22T20:33:16.012239+0000","last_became_active":"2026-02-22T20:33:14.007699+0000","last_became_peered":"2026-02-22T20:33:14.007699+0000","last_unstale":"2026-02-22T20:33:16.012239+0000","last_undegraded":"2026-02-22T20:33:16.012239+0000","last_fullsized":"2026-02-22T20:33:16.012239+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:23:43.192749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.183475+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:16.183475+0000","last_peered":"2026-02-22T20:33:16.183475+0000","last_clean":"2026-02-22T20:33:16.183475+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:16.183475+0000","last_undegraded":"2026-02-22T20:33:16.183475+0000","last_fullsized":"2026-02-22T20:33:16.183475+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.012057+0000","last_change":"2026-02-22T20:33:16.012159+0000","last_active":"2026-02-22T20:33:16.012057+0000","last_peered":"2026-02-22T20:33:16.012057+0000","last_clean":"2026-02-22T20:33:16.012057+0000","last_became_active":"2026-02-22T20:33:14.007350+0000","last_became_peered":"2026-02-22T20:33:14.007350+0000","last_unstale":"2026-02-22T20:33:16.012057+0000","last_undegraded":"2026-02-22T20:33:16.012057+0000","last_fullsized":"2026-02-22T20:33:16.012057+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:58:48.025926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.012474+0000","last_change":"2026-02-22T20:33:16.012704+0000","last_active":"2026-02-22T20:33:16.012474+0000","last_peered":"2026-02-22T20:33:16.012474+0000","last_clean":"2026-02-22T20:33:16.012474+0000","last_became_active":"2026-02-22T20:33:14.007293+0000","last_became_peered":"2026-02-22T20:33:14.007293+0000","last_unstale":"2026-02-22T20:33:16.012474+0000","last_undegraded":"2026-02-22T20:33:16.012474+0000","last_fullsized":"2026-02-22T20:33:16.012474+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:49:33.595619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052419200000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.013914+0000","last_change":"2026-02-22T20:33:16.013914+0000","last_active":"2026-02-22T20:33:16.013914+0000","last_peered":"2026-02-22T20:33:16.013914+0000","last_clean":"2026-02-22T20:33:16.013914+0000","last_became_active":"2026-02-22T20:33:14.007287+0000","last_became_peered":"2026-02-22T20:33:14.007287+0000","last_unstale":"2026-02-22T20:33:16.013914+0000","last_undegraded":"2026-02-22T20:33:16.013914+0000","last_fullsized":"2026-02-22T20:33:16.013914+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.011974+0000","last_change":"2026-02-22T20:33:16.012301+0000","last_active":"2026-02-22T20:33:16.011974+0000","last_peered":"2026-02-22T20:33:16.011974+0000","last_clean":"2026-02-22T20:33:16.011974+0000","last_became_active":"2026-02-22T20:33:14.007683+0000","last_became_peered":"2026-02-22T20:33:14.007683+0000","last_unstale":"2026-02-22T20:33:16.011974+0000","last_undegraded":"2026-02-22T20:33:16.011974+0000","last_fullsized":"2026-02-22T20:33:16.011974+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:02:07.868866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.012225+0000","last_change":"2026-02-22T20:33:16.012435+0000","last_active":"2026-02-22T20:33:16.012225+0000","last_peered":"2026-02-22T20:33:16.012225+0000","last_clean":"2026-02-22T20:33:16.012225+0000","last_became_active":"2026-02-22T20:33:14.007572+0000","last_became_peered":"2026-02-22T20:33:14.007572+0000","last_unstale":"2026-02-22T20:33:16.012225+0000","last_undegraded":"2026-02-22T20:33:16.012225+0000","last_fullsized":"2026-02-22T20:33:16.012225+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:28:46.662856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040373900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":112,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":114688,"data_stored":31457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6823,"internal_metadata":27583833},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27064,"kb_used_data":104,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344776,"statfs":{"total":96636764160,"available":96609050624,"internally_reserved":0,"allocated":106496,"data_stored":27850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6168,"internal_metadata":27584488},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":31476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27068,"kb_used_data":108,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344772,"statfs":{"total":96636764160,"available":96609046528,"internally_reserved":0,"allocated":110592,"data_stored":27858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6168,"internal_metadata":27584488},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:17.399 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:17.576 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:17.577 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:17.589 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2026-02-22T20:33:16.984141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":108280,"kb_used_data":440,"kb_used_omap":25,"kb_used_meta":107750,"kb_avail":377379080,"statfs":{"total":386547056640,"available":386436177920,"internally_reserved":0,"allocated":450560,"data_stored":118641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25984,"internal_metadata":110336640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.977955"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.012268+0000","last_change":"2026-02-22T20:33:16.012562+0000","last_active":"2026-02-22T20:33:16.012268+0000","last_peered":"2026-02-22T20:33:16.012268+0000","last_clean":"2026-02-22T20:33:16.012268+0000","last_became_active":"2026-02-22T20:33:14.005292+0000","last_became_peered":"2026-02-22T20:33:14.005292+0000","last_unstale":"2026-02-22T20:33:16.012268+0000","last_undegraded":"2026-02-22T20:33:16.012268+0000","last_fullsized":"2026-02-22T20:33:16.012268+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:19:42.247294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045825,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.012239+0000","last_change":"2026-02-22T20:33:16.012620+0000","last_active":"2026-02-22T20:33:16.012239+0000","last_peered":"2026-02-22T20:33:16.012239+0000","last_clean":"2026-02-22T20:33:16.012239+0000","last_became_active":"2026-02-22T20:33:14.007699+0000","last_became_peered":"2026-02-22T20:33:14.007699+0000","last_unstale":"2026-02-22T20:33:16.012239+0000","last_undegraded":"2026-02-22T20:33:16.012239+0000","last_fullsized":"2026-02-22T20:33:16.012239+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:23:43.192749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.183475+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:16.183475+0000","last_peered":"2026-02-22T20:33:16.183475+0000","last_clean":"2026-02-22T20:33:16.183475+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:16.183475+0000","last_undegraded":"2026-02-22T20:33:16.183475+0000","last_fullsized":"2026-02-22T20:33:16.183475+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.012057+0000","last_change":"2026-02-22T20:33:16.012159+0000","last_active":"2026-02-22T20:33:16.012057+0000","last_peered":"2026-02-22T20:33:16.012057+0000","last_clean":"2026-02-22T20:33:16.012057+0000","last_became_active":"2026-02-22T20:33:14.007350+0000","last_became_peered":"2026-02-22T20:33:14.007350+0000","last_unstale":"2026-02-22T20:33:16.012057+0000","last_undegraded":"2026-02-22T20:33:16.012057+0000","last_fullsized":"2026-02-22T20:33:16.012057+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:58:48.025926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.012474+0000","last_change":"2026-02-22T20:33:16.012704+0000","last_active":"2026-02-22T20:33:16.012474+0000","last_peered":"2026-02-22T20:33:16.012474+0000","last_clean":"2026-02-22T20:33:16.012474+0000","last_became_active":"2026-02-22T20:33:14.007293+0000","last_became_peered":"2026-02-22T20:33:14.007293+0000","last_unstale":"2026-02-22T20:33:16.012474+0000","last_undegraded":"2026-02-22T20:33:16.012474+0000","last_fullsized":"2026-02-22T20:33:16.012474+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:49:33.595619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052419200000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.013914+0000","last_change":"2026-02-22T20:33:16.013914+0000","last_active":"2026-02-22T20:33:16.013914+0000","last_peered":"2026-02-22T20:33:16.013914+0000","last_clean":"2026-02-22T20:33:16.013914+0000","last_became_active":"2026-02-22T20:33:14.007287+0000","last_became_peered":"2026-02-22T20:33:14.007287+0000","last_unstale":"2026-02-22T20:33:16.013914+0000","last_undegraded":"2026-02-22T20:33:16.013914+0000","last_fullsized":"2026-02-22T20:33:16.013914+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.011974+0000","last_change":"2026-02-22T20:33:16.012301+0000","last_active":"2026-02-22T20:33:16.011974+0000","last_peered":"2026-02-22T20:33:16.011974+0000","last_clean":"2026-02-22T20:33:16.011974+0000","last_became_active":"2026-02-22T20:33:14.007683+0000","last_became_peered":"2026-02-22T20:33:14.007683+0000","last_unstale":"2026-02-22T20:33:16.011974+0000","last_undegraded":"2026-02-22T20:33:16.011974+0000","last_fullsized":"2026-02-22T20:33:16.011974+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:02:07.868866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.012225+0000","last_change":"2026-02-22T20:33:16.012435+0000","last_active":"2026-02-22T20:33:16.012225+0000","last_peered":"2026-02-22T20:33:16.012225+0000","last_clean":"2026-02-22T20:33:16.012225+0000","last_became_active":"2026-02-22T20:33:14.007572+0000","last_became_peered":"2026-02-22T20:33:14.007572+0000","last_unstale":"2026-02-22T20:33:16.012225+0000","last_undegraded":"2026-02-22T20:33:16.012225+0000","last_fullsized":"2026-02-22T20:33:16.012225+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:28:46.662856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040373900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":112,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":114688,"data_stored":31457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6823,"internal_metadata":27583833},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27064,"kb_used_data":104,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344776,"statfs":{"total":96636764160,"available":96609050624,"internally_reserved":0,"allocated":106496,"data_stored":27850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6168,"internal_metadata":27584488},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":31476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27068,"kb_used_data":108,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344772,"statfs":{"total":96636764160,"available":96609046528,"internally_reserved":0,"allocated":110592,"data_stored":27858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6168,"internal_metadata":27584488},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:17.590 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-22T20:33:17.590 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-22T20:33:17.591 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-22T20:33:17.591 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T20:33:17.793 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:17.793 INFO:teuthology.orchestra.run.trial084.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-22T20:33:17.806 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-22T20:33:17.806 INFO:teuthology.run_tasks:Running task osd_recovery.test_incomplete_pgs... 2026-02-22T20:33:17.825 INFO:tasks.osd_recovery:num_osds is 4 2026-02-22T20:33:17.825 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T20:33:18.002 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:18.002 INFO:teuthology.orchestra.run.trial084.stdout:{"epoch":13,"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","created":"2026-02-22T20:33:03.270442+0000","modified":"2026-02-22T20:33:16.007814+0000","last_up_change":"2026-02-22T20:33:08.988844+0000","last_in_change":"2026-02-22T20:33:05.214562+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-22T20:33:12.644417+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca475199-167f-4aba-960a-1e3782214a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6802","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6803","nonce":1867649424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6804","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6805","nonce":1867649424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6808","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6809","nonce":1867649424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6806","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6807","nonce":1867649424}]},"public_addr":"10.20.193.84:6803/1867649424","cluster_addr":"10.20.193.84:6805/1867649424","heartbeat_back_addr":"10.20.193.84:6809/1867649424","heartbeat_front_addr":"10.20.193.84:6807/1867649424","state":["exists","up"]},{"osd":1,"uuid":"32df476a-2b69-493e-a636-1ba8bf5268a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6811","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6813","nonce":2769530292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6815","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6818","nonce":2769530292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6824","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6825","nonce":2769530292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6820","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6822","nonce":2769530292}]},"public_addr":"10.20.193.84:6813/2769530292","cluster_addr":"10.20.193.84:6818/2769530292","heartbeat_back_addr":"10.20.193.84:6825/2769530292","heartbeat_front_addr":"10.20.193.84:6822/2769530292","state":["exists","up"]},{"osd":2,"uuid":"196c3661-b3be-46a3-9f14-7908d9003447","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6826","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6827","nonce":2089419628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6828","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6829","nonce":2089419628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6832","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6833","nonce":2089419628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6830","nonce":2089419628},{"type":"v1","addr":"10.20.193.84:6831","nonce":2089419628}]},"public_addr":"10.20.193.84:6827/2089419628","cluster_addr":"10.20.193.84:6829/2089419628","heartbeat_back_addr":"10.20.193.84:6833/2089419628","heartbeat_front_addr":"10.20.193.84:6831/2089419628","state":["exists","up"]},{"osd":3,"uuid":"bdc6271f-361f-4362-b050-080c613ab56c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6810","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6812","nonce":2227945027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6814","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6816","nonce":2227945027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6821","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6823","nonce":2227945027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6817","nonce":2227945027},{"type":"v1","addr":"10.20.193.84:6819","nonce":2227945027}]},"public_addr":"10.20.193.84:6812/2227945027","cluster_addr":"10.20.193.84:6816/2227945027","heartbeat_back_addr":"10.20.193.84:6823/2227945027","heartbeat_front_addr":"10.20.193.84:6819/2227945027","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T20:33:18.014 INFO:tasks.osd_recovery.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2026-02-22T20:33:12.644417+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 13, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2026-02-22T20:33:18.015 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2026-02-22T20:33:18.195 INFO:teuthology.orchestra.run.trial084.stdout:pg_num: 8 2026-02-22T20:33:18.208 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2026-02-22T20:33:18.387 INFO:teuthology.orchestra.run.trial084.stdout:epoch 14 2026-02-22T20:33:18.388 INFO:teuthology.orchestra.run.trial084.stdout:fsid a4ea559c-fa00-404b-b21c-f34ee9bd0795 2026-02-22T20:33:18.388 INFO:teuthology.orchestra.run.trial084.stdout:created 2026-02-22T20:33:03.270442+0000 2026-02-22T20:33:18.388 INFO:teuthology.orchestra.run.trial084.stdout:modified 2026-02-22T20:33:18.057060+0000 2026-02-22T20:33:18.388 INFO:teuthology.orchestra.run.trial084.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2026-02-22T20:33:18.388 INFO:teuthology.orchestra.run.trial084.stdout:crush_version 4 2026-02-22T20:33:18.388 INFO:teuthology.orchestra.run.trial084.stdout:full_ratio 0.95 2026-02-22T20:33:18.388 INFO:teuthology.orchestra.run.trial084.stdout:backfillfull_ratio 0.9 2026-02-22T20:33:18.388 INFO:teuthology.orchestra.run.trial084.stdout:nearfull_ratio 0.85 2026-02-22T20:33:18.388 INFO:teuthology.orchestra.run.trial084.stdout:require_min_compat_client luminous 2026-02-22T20:33:18.389 INFO:teuthology.orchestra.run.trial084.stdout:min_compat_client jewel 2026-02-22T20:33:18.389 INFO:teuthology.orchestra.run.trial084.stdout:require_osd_release tentacle 2026-02-22T20:33:18.389 INFO:teuthology.orchestra.run.trial084.stdout:stretch_mode_enabled false 2026-02-22T20:33:18.389 INFO:teuthology.orchestra.run.trial084.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 13 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.50 2026-02-22T20:33:18.389 INFO:teuthology.orchestra.run.trial084.stdout:max_osd 4 2026-02-22T20:33:18.389 INFO:teuthology.orchestra.run.trial084.stdout:osd.0 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.84:6802/1867649424,v1:10.20.193.84:6803/1867649424] [v2:10.20.193.84:6804/1867649424,v1:10.20.193.84:6805/1867649424] exists,up ca475199-167f-4aba-960a-1e3782214a73 2026-02-22T20:33:18.389 INFO:teuthology.orchestra.run.trial084.stdout:osd.1 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.84:6811/2769530292,v1:10.20.193.84:6813/2769530292] [v2:10.20.193.84:6815/2769530292,v1:10.20.193.84:6818/2769530292] exists,up 32df476a-2b69-493e-a636-1ba8bf5268a2 2026-02-22T20:33:18.389 INFO:teuthology.orchestra.run.trial084.stdout:osd.2 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.84:6826/2089419628,v1:10.20.193.84:6827/2089419628] [v2:10.20.193.84:6828/2089419628,v1:10.20.193.84:6829/2089419628] exists,up 196c3661-b3be-46a3-9f14-7908d9003447 2026-02-22T20:33:18.389 INFO:teuthology.orchestra.run.trial084.stdout:osd.3 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.84:6810/2227945027,v1:10.20.193.84:6812/2227945027] [v2:10.20.193.84:6814/2227945027,v1:10.20.193.84:6816/2227945027] exists,up bdc6271f-361f-4362-b050-080c613ab56c 2026-02-22T20:33:18.400 INFO:tasks.osd_recovery.ceph_manager:['osd.0 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.84:6802/1867649424,v1:10.20.193.84:6803/1867649424] [v2:10.20.193.84:6804/1867649424,v1:10.20.193.84:6805/1867649424] exists,up ca475199-167f-4aba-960a-1e3782214a73', 'osd.1 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.84:6811/2769530292,v1:10.20.193.84:6813/2769530292] [v2:10.20.193.84:6815/2769530292,v1:10.20.193.84:6818/2769530292] exists,up 32df476a-2b69-493e-a636-1ba8bf5268a2', 'osd.2 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.84:6826/2089419628,v1:10.20.193.84:6827/2089419628] [v2:10.20.193.84:6828/2089419628,v1:10.20.193.84:6829/2089419628] exists,up 196c3661-b3be-46a3-9f14-7908d9003447', 'osd.3 up in weight 1 up_from 9 up_thru 10 down_at 0 last_clean_interval [0,0) [v2:10.20.193.84:6810/2227945027,v1:10.20.193.84:6812/2227945027] [v2:10.20.193.84:6814/2227945027,v1:10.20.193.84:6816/2227945027] exists,up bdc6271f-361f-4362-b050-080c613ab56c'] 2026-02-22T20:33:18.401 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-22T20:33:18.401 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-22T20:33:18.402 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-22T20:33:18.402 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-22T20:33:18.511 INFO:teuthology.orchestra.run.trial084.stdout:38654705669 2026-02-22T20:33:18.511 INFO:teuthology.orchestra.run.trial084.stdout:38654705670 2026-02-22T20:33:18.511 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-22T20:33:18.511 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T20:33:18.514 INFO:teuthology.orchestra.run.trial084.stdout:38654705668 2026-02-22T20:33:18.514 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T20:33:18.515 INFO:teuthology.orchestra.run.trial084.stdout:38654705669 2026-02-22T20:33:18.516 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T20:33:18.695 INFO:teuthology.orchestra.run.trial084.stdout:38654705669 2026-02-22T20:33:18.696 INFO:teuthology.orchestra.run.trial084.stdout:38654705668 2026-02-22T20:33:18.710 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705670 got 38654705669 for osd.3 2026-02-22T20:33:18.710 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705669 got 38654705668 for osd.0 2026-02-22T20:33:18.738 INFO:teuthology.orchestra.run.trial084.stdout:38654705667 2026-02-22T20:33:18.741 INFO:teuthology.orchestra.run.trial084.stdout:38654705668 2026-02-22T20:33:18.751 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705668 got 38654705667 for osd.1 2026-02-22T20:33:18.755 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705669 got 38654705668 for osd.2 2026-02-22T20:33:19.710 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-22T20:33:19.711 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T20:33:19.753 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T20:33:19.755 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T20:33:19.893 INFO:teuthology.orchestra.run.trial084.stdout:38654705670 2026-02-22T20:33:19.894 INFO:teuthology.orchestra.run.trial084.stdout:38654705669 2026-02-22T20:33:19.907 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705670 got 38654705670 for osd.3 2026-02-22T20:33:19.908 DEBUG:teuthology.parallel:result is None 2026-02-22T20:33:19.909 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705669 got 38654705669 for osd.0 2026-02-22T20:33:19.909 DEBUG:teuthology.parallel:result is None 2026-02-22T20:33:19.936 INFO:teuthology.orchestra.run.trial084.stdout:38654705668 2026-02-22T20:33:19.938 INFO:teuthology.orchestra.run.trial084.stdout:38654705669 2026-02-22T20:33:19.949 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705668 got 38654705668 for osd.1 2026-02-22T20:33:19.949 DEBUG:teuthology.parallel:result is None 2026-02-22T20:33:19.952 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705669 got 38654705669 for osd.2 2026-02-22T20:33:19.952 DEBUG:teuthology.parallel:result is None 2026-02-22T20:33:19.952 INFO:tasks.osd_recovery.ceph_manager:waiting for clean 2026-02-22T20:33:19.953 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:20.169 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:20.169 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:20.180 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2026-02-22T20:33:18.984345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":108312,"kb_used_data":472,"kb_used_omap":28,"kb_used_meta":107747,"kb_avail":377379048,"statfs":{"total":386547056640,"available":386436145152,"internally_reserved":0,"allocated":483328,"data_stored":132342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28752,"internal_metadata":110333872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.978160"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.059305+0000","last_change":"2026-02-22T20:33:16.012562+0000","last_active":"2026-02-22T20:33:18.059305+0000","last_peered":"2026-02-22T20:33:18.059305+0000","last_clean":"2026-02-22T20:33:18.059305+0000","last_became_active":"2026-02-22T20:33:14.005292+0000","last_became_peered":"2026-02-22T20:33:14.005292+0000","last_unstale":"2026-02-22T20:33:18.059305+0000","last_undegraded":"2026-02-22T20:33:18.059305+0000","last_fullsized":"2026-02-22T20:33:18.059305+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:19:42.247294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045825,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.059071+0000","last_change":"2026-02-22T20:33:16.012620+0000","last_active":"2026-02-22T20:33:18.059071+0000","last_peered":"2026-02-22T20:33:18.059071+0000","last_clean":"2026-02-22T20:33:18.059071+0000","last_became_active":"2026-02-22T20:33:14.007699+0000","last_became_peered":"2026-02-22T20:33:14.007699+0000","last_unstale":"2026-02-22T20:33:18.059071+0000","last_undegraded":"2026-02-22T20:33:18.059071+0000","last_fullsized":"2026-02-22T20:33:18.059071+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:23:43.192749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.155753+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:18.155753+0000","last_peered":"2026-02-22T20:33:18.155753+0000","last_clean":"2026-02-22T20:33:18.155753+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:18.155753+0000","last_undegraded":"2026-02-22T20:33:18.155753+0000","last_fullsized":"2026-02-22T20:33:18.155753+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.059071+0000","last_change":"2026-02-22T20:33:16.012159+0000","last_active":"2026-02-22T20:33:18.059071+0000","last_peered":"2026-02-22T20:33:18.059071+0000","last_clean":"2026-02-22T20:33:18.059071+0000","last_became_active":"2026-02-22T20:33:14.007350+0000","last_became_peered":"2026-02-22T20:33:14.007350+0000","last_unstale":"2026-02-22T20:33:18.059071+0000","last_undegraded":"2026-02-22T20:33:18.059071+0000","last_fullsized":"2026-02-22T20:33:18.059071+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:58:48.025926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.059216+0000","last_change":"2026-02-22T20:33:16.012704+0000","last_active":"2026-02-22T20:33:18.059216+0000","last_peered":"2026-02-22T20:33:18.059216+0000","last_clean":"2026-02-22T20:33:18.059216+0000","last_became_active":"2026-02-22T20:33:14.007293+0000","last_became_peered":"2026-02-22T20:33:14.007293+0000","last_unstale":"2026-02-22T20:33:18.059216+0000","last_undegraded":"2026-02-22T20:33:18.059216+0000","last_fullsized":"2026-02-22T20:33:18.059216+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:49:33.595619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052419200000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.013914+0000","last_change":"2026-02-22T20:33:16.013914+0000","last_active":"2026-02-22T20:33:16.013914+0000","last_peered":"2026-02-22T20:33:16.013914+0000","last_clean":"2026-02-22T20:33:16.013914+0000","last_became_active":"2026-02-22T20:33:14.007287+0000","last_became_peered":"2026-02-22T20:33:14.007287+0000","last_unstale":"2026-02-22T20:33:16.013914+0000","last_undegraded":"2026-02-22T20:33:16.013914+0000","last_fullsized":"2026-02-22T20:33:16.013914+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.059071+0000","last_change":"2026-02-22T20:33:16.012301+0000","last_active":"2026-02-22T20:33:18.059071+0000","last_peered":"2026-02-22T20:33:18.059071+0000","last_clean":"2026-02-22T20:33:18.059071+0000","last_became_active":"2026-02-22T20:33:14.007683+0000","last_became_peered":"2026-02-22T20:33:14.007683+0000","last_unstale":"2026-02-22T20:33:18.059071+0000","last_undegraded":"2026-02-22T20:33:18.059071+0000","last_fullsized":"2026-02-22T20:33:18.059071+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:02:07.868866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.059206+0000","last_change":"2026-02-22T20:33:16.012435+0000","last_active":"2026-02-22T20:33:18.059206+0000","last_peered":"2026-02-22T20:33:18.059206+0000","last_clean":"2026-02-22T20:33:18.059206+0000","last_became_active":"2026-02-22T20:33:14.007572+0000","last_became_peered":"2026-02-22T20:33:14.007572+0000","last_unstale":"2026-02-22T20:33:18.059206+0000","last_undegraded":"2026-02-22T20:33:18.059206+0000","last_fullsized":"2026-02-22T20:33:18.059206+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:28:46.662856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040373900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27080,"kb_used_data":120,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344760,"statfs":{"total":96636764160,"available":96609034240,"internally_reserved":0,"allocated":122880,"data_stored":34695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7550,"internal_metadata":27583106},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":112,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":114688,"data_stored":31457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27084,"kb_used_data":124,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344756,"statfs":{"total":96636764160,"available":96609030144,"internally_reserved":0,"allocated":126976,"data_stored":34714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7552,"internal_metadata":27583104},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":31476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:20.182 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:20.356 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:20.357 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:20.369 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2026-02-22T20:33:18.984345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":108312,"kb_used_data":472,"kb_used_omap":28,"kb_used_meta":107747,"kb_avail":377379048,"statfs":{"total":386547056640,"available":386436145152,"internally_reserved":0,"allocated":483328,"data_stored":132342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28752,"internal_metadata":110333872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.978160"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.059305+0000","last_change":"2026-02-22T20:33:16.012562+0000","last_active":"2026-02-22T20:33:18.059305+0000","last_peered":"2026-02-22T20:33:18.059305+0000","last_clean":"2026-02-22T20:33:18.059305+0000","last_became_active":"2026-02-22T20:33:14.005292+0000","last_became_peered":"2026-02-22T20:33:14.005292+0000","last_unstale":"2026-02-22T20:33:18.059305+0000","last_undegraded":"2026-02-22T20:33:18.059305+0000","last_fullsized":"2026-02-22T20:33:18.059305+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:19:42.247294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045825,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.059071+0000","last_change":"2026-02-22T20:33:16.012620+0000","last_active":"2026-02-22T20:33:18.059071+0000","last_peered":"2026-02-22T20:33:18.059071+0000","last_clean":"2026-02-22T20:33:18.059071+0000","last_became_active":"2026-02-22T20:33:14.007699+0000","last_became_peered":"2026-02-22T20:33:14.007699+0000","last_unstale":"2026-02-22T20:33:18.059071+0000","last_undegraded":"2026-02-22T20:33:18.059071+0000","last_fullsized":"2026-02-22T20:33:18.059071+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:23:43.192749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.155753+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:18.155753+0000","last_peered":"2026-02-22T20:33:18.155753+0000","last_clean":"2026-02-22T20:33:18.155753+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:18.155753+0000","last_undegraded":"2026-02-22T20:33:18.155753+0000","last_fullsized":"2026-02-22T20:33:18.155753+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.059071+0000","last_change":"2026-02-22T20:33:16.012159+0000","last_active":"2026-02-22T20:33:18.059071+0000","last_peered":"2026-02-22T20:33:18.059071+0000","last_clean":"2026-02-22T20:33:18.059071+0000","last_became_active":"2026-02-22T20:33:14.007350+0000","last_became_peered":"2026-02-22T20:33:14.007350+0000","last_unstale":"2026-02-22T20:33:18.059071+0000","last_undegraded":"2026-02-22T20:33:18.059071+0000","last_fullsized":"2026-02-22T20:33:18.059071+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:58:48.025926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.059216+0000","last_change":"2026-02-22T20:33:16.012704+0000","last_active":"2026-02-22T20:33:18.059216+0000","last_peered":"2026-02-22T20:33:18.059216+0000","last_clean":"2026-02-22T20:33:18.059216+0000","last_became_active":"2026-02-22T20:33:14.007293+0000","last_became_peered":"2026-02-22T20:33:14.007293+0000","last_unstale":"2026-02-22T20:33:18.059216+0000","last_undegraded":"2026-02-22T20:33:18.059216+0000","last_fullsized":"2026-02-22T20:33:18.059216+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:49:33.595619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052419200000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":22,"reported_epoch":13,"state":"active+clean","last_fresh":"2026-02-22T20:33:16.013914+0000","last_change":"2026-02-22T20:33:16.013914+0000","last_active":"2026-02-22T20:33:16.013914+0000","last_peered":"2026-02-22T20:33:16.013914+0000","last_clean":"2026-02-22T20:33:16.013914+0000","last_became_active":"2026-02-22T20:33:14.007287+0000","last_became_peered":"2026-02-22T20:33:14.007287+0000","last_unstale":"2026-02-22T20:33:16.013914+0000","last_undegraded":"2026-02-22T20:33:16.013914+0000","last_fullsized":"2026-02-22T20:33:16.013914+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.059071+0000","last_change":"2026-02-22T20:33:16.012301+0000","last_active":"2026-02-22T20:33:18.059071+0000","last_peered":"2026-02-22T20:33:18.059071+0000","last_clean":"2026-02-22T20:33:18.059071+0000","last_became_active":"2026-02-22T20:33:14.007683+0000","last_became_peered":"2026-02-22T20:33:14.007683+0000","last_unstale":"2026-02-22T20:33:18.059071+0000","last_undegraded":"2026-02-22T20:33:18.059071+0000","last_fullsized":"2026-02-22T20:33:18.059071+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:02:07.868866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T20:33:18.059206+0000","last_change":"2026-02-22T20:33:16.012435+0000","last_active":"2026-02-22T20:33:18.059206+0000","last_peered":"2026-02-22T20:33:18.059206+0000","last_clean":"2026-02-22T20:33:18.059206+0000","last_became_active":"2026-02-22T20:33:14.007572+0000","last_became_peered":"2026-02-22T20:33:14.007572+0000","last_unstale":"2026-02-22T20:33:18.059206+0000","last_undegraded":"2026-02-22T20:33:18.059206+0000","last_fullsized":"2026-02-22T20:33:18.059206+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:28:46.662856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040373900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27080,"kb_used_data":120,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344760,"statfs":{"total":96636764160,"available":96609034240,"internally_reserved":0,"allocated":122880,"data_stored":34695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7550,"internal_metadata":27583106},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27072,"kb_used_data":112,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344768,"statfs":{"total":96636764160,"available":96609042432,"internally_reserved":0,"allocated":114688,"data_stored":31457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27084,"kb_used_data":124,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344756,"statfs":{"total":96636764160,"available":96609030144,"internally_reserved":0,"allocated":126976,"data_stored":34714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7552,"internal_metadata":27583104},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27076,"kb_used_data":116,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344764,"statfs":{"total":96636764160,"available":96609038336,"internally_reserved":0,"allocated":118784,"data_stored":31476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:20.370 INFO:tasks.osd_recovery.ceph_manager:clean! 2026-02-22T20:33:20.370 INFO:tasks.osd_recovery:Testing incomplete pgs... 2026-02-22T20:33:20.371 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set osd_recovery_delay_start 1000 2026-02-22T20:33:20.455 INFO:teuthology.orchestra.run.trial084.stdout:{ 2026-02-22T20:33:20.455 INFO:teuthology.orchestra.run.trial084.stdout: "success": "osd_recovery_delay_start = '' " 2026-02-22T20:33:20.455 INFO:teuthology.orchestra.run.trial084.stdout:} 2026-02-22T20:33:20.462 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set osd_recovery_delay_start 1000 2026-02-22T20:33:20.547 INFO:teuthology.orchestra.run.trial084.stdout:{ 2026-02-22T20:33:20.548 INFO:teuthology.orchestra.run.trial084.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_delay_start = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-22T20:33:20.548 INFO:teuthology.orchestra.run.trial084.stdout:} 2026-02-22T20:33:20.555 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set osd_recovery_delay_start 1000 2026-02-22T20:33:20.639 INFO:teuthology.orchestra.run.trial084.stdout:{ 2026-02-22T20:33:20.640 INFO:teuthology.orchestra.run.trial084.stdout: "success": "osd_recovery_delay_start = '' " 2026-02-22T20:33:20.640 INFO:teuthology.orchestra.run.trial084.stdout:} 2026-02-22T20:33:20.648 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set osd_recovery_delay_start 1000 2026-02-22T20:33:20.733 INFO:teuthology.orchestra.run.trial084.stdout:{ 2026-02-22T20:33:20.733 INFO:teuthology.orchestra.run.trial084.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_delay_start = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-22T20:33:20.733 INFO:teuthology.orchestra.run.trial084.stdout:} 2026-02-22T20:33:20.741 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 1 2026-02-22T20:33:20.923 INFO:tasks.ceph.mon.a.trial084.stderr:2026-02-22T20:33:20.921+0000 7fb50627a6c0 -1 mon.a@0(leader).osd e14 definitely_dead 0 2026-02-22T20:33:21.070 INFO:teuthology.orchestra.run.trial084.stderr:marked out osd.0. marked out osd.1. 2026-02-22T20:33:21.087 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-22T20:33:21.087 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-22T20:33:21.087 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-22T20:33:21.087 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-22T20:33:21.197 INFO:teuthology.orchestra.run.trial084.stdout:38654705672 2026-02-22T20:33:21.197 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-22T20:33:21.201 INFO:teuthology.orchestra.run.trial084.stdout:38654705670 2026-02-22T20:33:21.201 DEBUG:teuthology.parallel:result is None 2026-02-22T20:33:21.202 INFO:teuthology.orchestra.run.trial084.stdout:38654705671 2026-02-22T20:33:21.202 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T20:33:21.203 INFO:teuthology.orchestra.run.trial084.stdout:38654705671 2026-02-22T20:33:21.203 DEBUG:teuthology.parallel:result is None 2026-02-22T20:33:21.379 INFO:teuthology.orchestra.run.trial084.stdout:38654705671 2026-02-22T20:33:21.393 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705672 got 38654705671 for osd.3 2026-02-22T20:33:21.421 INFO:teuthology.orchestra.run.trial084.stdout:38654705670 2026-02-22T20:33:21.434 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705671 got 38654705670 for osd.2 2026-02-22T20:33:22.394 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-22T20:33:22.435 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T20:33:22.577 INFO:teuthology.orchestra.run.trial084.stdout:38654705671 2026-02-22T20:33:22.591 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705672 got 38654705671 for osd.3 2026-02-22T20:33:22.615 INFO:teuthology.orchestra.run.trial084.stdout:38654705670 2026-02-22T20:33:22.628 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705671 got 38654705670 for osd.2 2026-02-22T20:33:23.593 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-22T20:33:23.629 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T20:33:23.778 INFO:teuthology.orchestra.run.trial084.stdout:38654705672 2026-02-22T20:33:23.794 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705672 got 38654705672 for osd.3 2026-02-22T20:33:23.794 DEBUG:teuthology.parallel:result is None 2026-02-22T20:33:23.814 INFO:teuthology.orchestra.run.trial084.stdout:38654705671 2026-02-22T20:33:23.827 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705671 got 38654705671 for osd.2 2026-02-22T20:33:23.827 DEBUG:teuthology.parallel:result is None 2026-02-22T20:33:23.827 INFO:tasks.osd_recovery.ceph_manager:waiting for clean 2026-02-22T20:33:23.828 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:24.043 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:24.044 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:24.056 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2026-02-22T20:33:22.984831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54160,"kb_used_data":240,"kb_used_omap":14,"kb_used_meta":53873,"kb_avail":188689520,"statfs":{"total":193273528320,"available":193218068480,"internally_reserved":0,"allocated":245760,"data_stored":69390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15102,"internal_metadata":55166210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000690"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":3,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:21.071000+0000","last_change":"2026-02-22T20:33:21.071000+0000","last_active":"2026-02-22T20:33:12.997318+0000","last_peered":"2026-02-22T20:33:12.997318+0000","last_clean":"2026-02-22T20:33:12.997318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:33:21.071000+0000","last_undegraded":"2026-02-22T20:33:21.071000+0000","last_fullsized":"2026-02-22T20:33:21.071000+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":14,"state":"peering","last_fresh":"2026-02-22T20:33:18.059098+0000","last_change":"2026-02-22T20:33:21.070965+0000","last_active":"2026-02-22T20:33:21.070965+0000","last_peered":"2026-02-22T20:33:18.059098+0000","last_clean":"2026-02-22T20:33:18.059098+0000","last_became_active":"2026-02-22T20:33:14.007699+0000","last_became_peered":"2026-02-22T20:33:14.007699+0000","last_unstale":"2026-02-22T20:33:18.059098+0000","last_undegraded":"2026-02-22T20:33:18.059098+0000","last_fullsized":"2026-02-22T20:33:18.059098+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":24,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T20:33:21.072688+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:21.072688+0000","last_peered":"2026-02-22T20:33:21.072688+0000","last_clean":"2026-02-22T20:33:21.072688+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:21.072688+0000","last_undegraded":"2026-02-22T20:33:21.072688+0000","last_fullsized":"2026-02-22T20:33:21.072688+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":23,"reported_epoch":14,"state":"peering","last_fresh":"2026-02-22T20:33:18.059102+0000","last_change":"2026-02-22T20:33:21.071358+0000","last_active":"2026-02-22T20:33:21.071358+0000","last_peered":"2026-02-22T20:33:18.059102+0000","last_clean":"2026-02-22T20:33:18.059102+0000","last_became_active":"2026-02-22T20:33:14.007350+0000","last_became_peered":"2026-02-22T20:33:14.007350+0000","last_unstale":"2026-02-22T20:33:18.059102+0000","last_undegraded":"2026-02-22T20:33:18.059102+0000","last_fullsized":"2026-02-22T20:33:18.059102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":10,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:21.072819+0000","last_change":"2026-02-22T20:33:21.072819+0000","last_active":"2026-02-22T20:33:21.072819+0000","last_peered":"2026-02-22T20:33:14.007552+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:14.007293+0000","last_became_peered":"2026-02-22T20:33:14.007293+0000","last_unstale":"2026-02-22T20:33:21.072819+0000","last_undegraded":"2026-02-22T20:33:21.072819+0000","last_fullsized":"2026-02-22T20:33:21.072819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":2,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:18.852969+0000","last_change":"2026-02-22T20:33:21.184030+0000","last_active":"2026-02-22T20:33:21.184030+0000","last_peered":"2026-02-22T20:33:18.852969+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:14.007287+0000","last_became_peered":"2026-02-22T20:33:14.007287+0000","last_unstale":"2026-02-22T20:33:18.852969+0000","last_undegraded":"2026-02-22T20:33:18.852969+0000","last_fullsized":"2026-02-22T20:33:18.852969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":14,"state":"peering","last_fresh":"2026-02-22T20:33:18.059137+0000","last_change":"2026-02-22T20:33:21.071028+0000","last_active":"2026-02-22T20:33:21.071028+0000","last_peered":"2026-02-22T20:33:18.059137+0000","last_clean":"2026-02-22T20:33:18.059137+0000","last_became_active":"2026-02-22T20:33:14.007683+0000","last_became_peered":"2026-02-22T20:33:14.007683+0000","last_unstale":"2026-02-22T20:33:18.059137+0000","last_undegraded":"2026-02-22T20:33:18.059137+0000","last_fullsized":"2026-02-22T20:33:18.059137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":2,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:14.003447+0000","last_change":"2026-02-22T20:33:21.185550+0000","last_active":"2026-02-22T20:33:12.997318+0000","last_peered":"2026-02-22T20:33:12.997318+0000","last_clean":"2026-02-22T20:33:12.997318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:33:14.003447+0000","last_undegraded":"2026-02-22T20:33:14.003447+0000","last_fullsized":"2026-02-22T20:33:14.003447+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705672,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27080,"kb_used_data":120,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344760,"statfs":{"total":96636764160,"available":96609034240,"internally_reserved":0,"allocated":122880,"data_stored":34695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7550,"internal_metadata":27583106},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27080,"kb_used_data":120,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344760,"statfs":{"total":96636764160,"available":96609034240,"internally_reserved":0,"allocated":122880,"data_stored":34695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7552,"internal_metadata":27583104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705670,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705671,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:24.057 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:24.235 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:24.235 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:24.247 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2026-02-22T20:33:22.984831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54160,"kb_used_data":240,"kb_used_omap":14,"kb_used_meta":53873,"kb_avail":188689520,"statfs":{"total":193273528320,"available":193218068480,"internally_reserved":0,"allocated":245760,"data_stored":69390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15102,"internal_metadata":55166210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000690"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":3,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:21.071000+0000","last_change":"2026-02-22T20:33:21.071000+0000","last_active":"2026-02-22T20:33:12.997318+0000","last_peered":"2026-02-22T20:33:12.997318+0000","last_clean":"2026-02-22T20:33:12.997318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:33:21.071000+0000","last_undegraded":"2026-02-22T20:33:21.071000+0000","last_fullsized":"2026-02-22T20:33:21.071000+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":14,"state":"peering","last_fresh":"2026-02-22T20:33:18.059098+0000","last_change":"2026-02-22T20:33:21.070965+0000","last_active":"2026-02-22T20:33:21.070965+0000","last_peered":"2026-02-22T20:33:18.059098+0000","last_clean":"2026-02-22T20:33:18.059098+0000","last_became_active":"2026-02-22T20:33:14.007699+0000","last_became_peered":"2026-02-22T20:33:14.007699+0000","last_unstale":"2026-02-22T20:33:18.059098+0000","last_undegraded":"2026-02-22T20:33:18.059098+0000","last_fullsized":"2026-02-22T20:33:18.059098+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":24,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T20:33:21.072688+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:21.072688+0000","last_peered":"2026-02-22T20:33:21.072688+0000","last_clean":"2026-02-22T20:33:21.072688+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:21.072688+0000","last_undegraded":"2026-02-22T20:33:21.072688+0000","last_fullsized":"2026-02-22T20:33:21.072688+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":23,"reported_epoch":14,"state":"peering","last_fresh":"2026-02-22T20:33:18.059102+0000","last_change":"2026-02-22T20:33:21.071358+0000","last_active":"2026-02-22T20:33:21.071358+0000","last_peered":"2026-02-22T20:33:18.059102+0000","last_clean":"2026-02-22T20:33:18.059102+0000","last_became_active":"2026-02-22T20:33:14.007350+0000","last_became_peered":"2026-02-22T20:33:14.007350+0000","last_unstale":"2026-02-22T20:33:18.059102+0000","last_undegraded":"2026-02-22T20:33:18.059102+0000","last_fullsized":"2026-02-22T20:33:18.059102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":10,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:21.072819+0000","last_change":"2026-02-22T20:33:21.072819+0000","last_active":"2026-02-22T20:33:21.072819+0000","last_peered":"2026-02-22T20:33:14.007552+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:14.007293+0000","last_became_peered":"2026-02-22T20:33:14.007293+0000","last_unstale":"2026-02-22T20:33:21.072819+0000","last_undegraded":"2026-02-22T20:33:21.072819+0000","last_fullsized":"2026-02-22T20:33:21.072819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":2,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:18.852969+0000","last_change":"2026-02-22T20:33:21.184030+0000","last_active":"2026-02-22T20:33:21.184030+0000","last_peered":"2026-02-22T20:33:18.852969+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:14.007287+0000","last_became_peered":"2026-02-22T20:33:14.007287+0000","last_unstale":"2026-02-22T20:33:18.852969+0000","last_undegraded":"2026-02-22T20:33:18.852969+0000","last_fullsized":"2026-02-22T20:33:18.852969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":14,"state":"peering","last_fresh":"2026-02-22T20:33:18.059137+0000","last_change":"2026-02-22T20:33:21.071028+0000","last_active":"2026-02-22T20:33:21.071028+0000","last_peered":"2026-02-22T20:33:18.059137+0000","last_clean":"2026-02-22T20:33:18.059137+0000","last_became_active":"2026-02-22T20:33:14.007683+0000","last_became_peered":"2026-02-22T20:33:14.007683+0000","last_unstale":"2026-02-22T20:33:18.059137+0000","last_undegraded":"2026-02-22T20:33:18.059137+0000","last_fullsized":"2026-02-22T20:33:18.059137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":2,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:14.003447+0000","last_change":"2026-02-22T20:33:21.185550+0000","last_active":"2026-02-22T20:33:12.997318+0000","last_peered":"2026-02-22T20:33:12.997318+0000","last_clean":"2026-02-22T20:33:12.997318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:33:14.003447+0000","last_undegraded":"2026-02-22T20:33:14.003447+0000","last_fullsized":"2026-02-22T20:33:14.003447+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705672,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27080,"kb_used_data":120,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344760,"statfs":{"total":96636764160,"available":96609034240,"internally_reserved":0,"allocated":122880,"data_stored":34695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7550,"internal_metadata":27583106},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27080,"kb_used_data":120,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344760,"statfs":{"total":96636764160,"available":96609034240,"internally_reserved":0,"allocated":122880,"data_stored":34695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7552,"internal_metadata":27583104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705670,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705671,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:24.248 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:24.424 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:24.425 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:24.436 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2026-02-22T20:33:22.984831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54160,"kb_used_data":240,"kb_used_omap":14,"kb_used_meta":53873,"kb_avail":188689520,"statfs":{"total":193273528320,"available":193218068480,"internally_reserved":0,"allocated":245760,"data_stored":69390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15102,"internal_metadata":55166210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000690"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":3,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:21.071000+0000","last_change":"2026-02-22T20:33:21.071000+0000","last_active":"2026-02-22T20:33:12.997318+0000","last_peered":"2026-02-22T20:33:12.997318+0000","last_clean":"2026-02-22T20:33:12.997318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:33:21.071000+0000","last_undegraded":"2026-02-22T20:33:21.071000+0000","last_fullsized":"2026-02-22T20:33:21.071000+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":14,"state":"peering","last_fresh":"2026-02-22T20:33:18.059098+0000","last_change":"2026-02-22T20:33:21.070965+0000","last_active":"2026-02-22T20:33:21.070965+0000","last_peered":"2026-02-22T20:33:18.059098+0000","last_clean":"2026-02-22T20:33:18.059098+0000","last_became_active":"2026-02-22T20:33:14.007699+0000","last_became_peered":"2026-02-22T20:33:14.007699+0000","last_unstale":"2026-02-22T20:33:18.059098+0000","last_undegraded":"2026-02-22T20:33:18.059098+0000","last_fullsized":"2026-02-22T20:33:18.059098+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":24,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T20:33:21.072688+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:21.072688+0000","last_peered":"2026-02-22T20:33:21.072688+0000","last_clean":"2026-02-22T20:33:21.072688+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:21.072688+0000","last_undegraded":"2026-02-22T20:33:21.072688+0000","last_fullsized":"2026-02-22T20:33:21.072688+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":23,"reported_epoch":14,"state":"peering","last_fresh":"2026-02-22T20:33:18.059102+0000","last_change":"2026-02-22T20:33:21.071358+0000","last_active":"2026-02-22T20:33:21.071358+0000","last_peered":"2026-02-22T20:33:18.059102+0000","last_clean":"2026-02-22T20:33:18.059102+0000","last_became_active":"2026-02-22T20:33:14.007350+0000","last_became_peered":"2026-02-22T20:33:14.007350+0000","last_unstale":"2026-02-22T20:33:18.059102+0000","last_undegraded":"2026-02-22T20:33:18.059102+0000","last_fullsized":"2026-02-22T20:33:18.059102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":10,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:21.072819+0000","last_change":"2026-02-22T20:33:21.072819+0000","last_active":"2026-02-22T20:33:21.072819+0000","last_peered":"2026-02-22T20:33:14.007552+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:14.007293+0000","last_became_peered":"2026-02-22T20:33:14.007293+0000","last_unstale":"2026-02-22T20:33:21.072819+0000","last_undegraded":"2026-02-22T20:33:21.072819+0000","last_fullsized":"2026-02-22T20:33:21.072819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":2,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:18.852969+0000","last_change":"2026-02-22T20:33:21.184030+0000","last_active":"2026-02-22T20:33:21.184030+0000","last_peered":"2026-02-22T20:33:18.852969+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:14.007287+0000","last_became_peered":"2026-02-22T20:33:14.007287+0000","last_unstale":"2026-02-22T20:33:18.852969+0000","last_undegraded":"2026-02-22T20:33:18.852969+0000","last_fullsized":"2026-02-22T20:33:18.852969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":14,"state":"peering","last_fresh":"2026-02-22T20:33:18.059137+0000","last_change":"2026-02-22T20:33:21.071028+0000","last_active":"2026-02-22T20:33:21.071028+0000","last_peered":"2026-02-22T20:33:18.059137+0000","last_clean":"2026-02-22T20:33:18.059137+0000","last_became_active":"2026-02-22T20:33:14.007683+0000","last_became_peered":"2026-02-22T20:33:14.007683+0000","last_unstale":"2026-02-22T20:33:18.059137+0000","last_undegraded":"2026-02-22T20:33:18.059137+0000","last_fullsized":"2026-02-22T20:33:18.059137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":2,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:14.003447+0000","last_change":"2026-02-22T20:33:21.185550+0000","last_active":"2026-02-22T20:33:12.997318+0000","last_peered":"2026-02-22T20:33:12.997318+0000","last_clean":"2026-02-22T20:33:12.997318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:33:14.003447+0000","last_undegraded":"2026-02-22T20:33:14.003447+0000","last_fullsized":"2026-02-22T20:33:14.003447+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705672,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27080,"kb_used_data":120,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344760,"statfs":{"total":96636764160,"available":96609034240,"internally_reserved":0,"allocated":122880,"data_stored":34695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7550,"internal_metadata":27583106},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27080,"kb_used_data":120,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344760,"statfs":{"total":96636764160,"available":96609034240,"internally_reserved":0,"allocated":122880,"data_stored":34695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7552,"internal_metadata":27583104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705670,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705671,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 15, 'state': 'peering', 'last_fresh': '2026-02-22T20:33:21.071000+0000', 'last_change': '2026-02-22T20:33:21.071000+0000', 'last_active': '2026-02-22T20:33:12.997318+0000', 'last_peered': '2026-02-22T20:33:12.997318+0000', 'last_clean': '2026-02-22T20:33:12.997318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2026-02-22T20:33:21.071000+0000', 'last_undegraded': '2026-02-22T20:33:21.071000+0000', 'last_fullsized': '2026-02-22T20:33:21.071000+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "0'0", 'reported_seq': 23, 'reported_epoch': 14, 'state': 'peering', 'last_fresh': '2026-02-22T20:33:18.059098+0000', 'last_change': '2026-02-22T20:33:21.070965+0000', 'last_active': '2026-02-22T20:33:21.070965+0000', 'last_peered': '2026-02-22T20:33:18.059098+0000', 'last_clean': '2026-02-22T20:33:18.059098+0000', 'last_became_active': '2026-02-22T20:33:14.007699+0000', 'last_became_peered': '2026-02-22T20:33:14.007699+0000', 'last_unstale': '2026-02-22T20:33:18.059098+0000', 'last_undegraded': '2026-02-22T20:33:18.059098+0000', 'last_fullsized': '2026-02-22T20:33:18.059098+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "0'0", 'reported_seq': 23, 'reported_epoch': 14, 'state': 'peering', 'last_fresh': '2026-02-22T20:33:18.059102+0000', 'last_change': '2026-02-22T20:33:21.071358+0000', 'last_active': '2026-02-22T20:33:21.071358+0000', 'last_peered': '2026-02-22T20:33:18.059102+0000', 'last_clean': '2026-02-22T20:33:18.059102+0000', 'last_became_active': '2026-02-22T20:33:14.007350+0000', 'last_became_peered': '2026-02-22T20:33:14.007350+0000', 'last_unstale': '2026-02-22T20:33:18.059102+0000', 'last_undegraded': '2026-02-22T20:33:18.059102+0000', 'last_fullsized': '2026-02-22T20:33:18.059102+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 10, 'reported_epoch': 15, 'state': 'peering', 'last_fresh': '2026-02-22T20:33:21.072819+0000', 'last_change': '2026-02-22T20:33:21.072819+0000', 'last_active': '2026-02-22T20:33:21.072819+0000', 'last_peered': '2026-02-22T20:33:14.007552+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:14.007293+0000', 'last_became_peered': '2026-02-22T20:33:14.007293+0000', 'last_unstale': '2026-02-22T20:33:21.072819+0000', 'last_undegraded': '2026-02-22T20:33:21.072819+0000', 'last_fullsized': '2026-02-22T20:33:21.072819+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 2, 'reported_epoch': 15, 'state': 'peering', 'last_fresh': '2026-02-22T20:33:18.852969+0000', 'last_change': '2026-02-22T20:33:21.184030+0000', 'last_active': '2026-02-22T20:33:21.184030+0000', 'last_peered': '2026-02-22T20:33:18.852969+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:14.007287+0000', 'last_became_peered': '2026-02-22T20:33:14.007287+0000', 'last_unstale': '2026-02-22T20:33:18.852969+0000', 'last_undegraded': '2026-02-22T20:33:18.852969+0000', 'last_fullsized': '2026-02-22T20:33:18.852969+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "0'0", 'reported_seq': 23, 'reported_epoch': 14, 'state': 'peering', 'last_fresh': '2026-02-22T20:33:18.059137+0000', 'last_change': '2026-02-22T20:33:21.071028+0000', 'last_active': '2026-02-22T20:33:21.071028+0000', 'last_peered': '2026-02-22T20:33:18.059137+0000', 'last_clean': '2026-02-22T20:33:18.059137+0000', 'last_became_active': '2026-02-22T20:33:14.007683+0000', 'last_became_peered': '2026-02-22T20:33:14.007683+0000', 'last_unstale': '2026-02-22T20:33:18.059137+0000', 'last_undegraded': '2026-02-22T20:33:18.059137+0000', 'last_fullsized': '2026-02-22T20:33:18.059137+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:33:24.437 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 15, 'state': 'peering', 'last_fresh': '2026-02-22T20:33:14.003447+0000', 'last_change': '2026-02-22T20:33:21.185550+0000', 'last_active': '2026-02-22T20:33:12.997318+0000', 'last_peered': '2026-02-22T20:33:12.997318+0000', 'last_clean': '2026-02-22T20:33:12.997318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2026-02-22T20:33:14.003447+0000', 'last_undegraded': '2026-02-22T20:33:14.003447+0000', 'last_fullsized': '2026-02-22T20:33:14.003447+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:33:24.437 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:33:24.653 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:24.653 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":21,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":16,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":7},{"state_name":"active+clean","count":1}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55459840,"bytes_avail":193218068480,"bytes_total":193273528320,"inactive_pgs_ratio":0.875},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-22T20:33:03.271077+0000","services":{}},"progress_events":{}} 2026-02-22T20:33:24.666 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:33:24.666 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:24.844 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:24.845 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:24.856 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2026-02-22T20:33:22.984831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54160,"kb_used_data":240,"kb_used_omap":14,"kb_used_meta":53873,"kb_avail":188689520,"statfs":{"total":193273528320,"available":193218068480,"internally_reserved":0,"allocated":245760,"data_stored":69390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15102,"internal_metadata":55166210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000690"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":3,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:21.071000+0000","last_change":"2026-02-22T20:33:21.071000+0000","last_active":"2026-02-22T20:33:12.997318+0000","last_peered":"2026-02-22T20:33:12.997318+0000","last_clean":"2026-02-22T20:33:12.997318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:33:21.071000+0000","last_undegraded":"2026-02-22T20:33:21.071000+0000","last_fullsized":"2026-02-22T20:33:21.071000+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":14,"state":"peering","last_fresh":"2026-02-22T20:33:18.059098+0000","last_change":"2026-02-22T20:33:21.070965+0000","last_active":"2026-02-22T20:33:21.070965+0000","last_peered":"2026-02-22T20:33:18.059098+0000","last_clean":"2026-02-22T20:33:18.059098+0000","last_became_active":"2026-02-22T20:33:14.007699+0000","last_became_peered":"2026-02-22T20:33:14.007699+0000","last_unstale":"2026-02-22T20:33:18.059098+0000","last_undegraded":"2026-02-22T20:33:18.059098+0000","last_fullsized":"2026-02-22T20:33:18.059098+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":24,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T20:33:21.072688+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:21.072688+0000","last_peered":"2026-02-22T20:33:21.072688+0000","last_clean":"2026-02-22T20:33:21.072688+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:21.072688+0000","last_undegraded":"2026-02-22T20:33:21.072688+0000","last_fullsized":"2026-02-22T20:33:21.072688+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":23,"reported_epoch":14,"state":"peering","last_fresh":"2026-02-22T20:33:18.059102+0000","last_change":"2026-02-22T20:33:21.071358+0000","last_active":"2026-02-22T20:33:21.071358+0000","last_peered":"2026-02-22T20:33:18.059102+0000","last_clean":"2026-02-22T20:33:18.059102+0000","last_became_active":"2026-02-22T20:33:14.007350+0000","last_became_peered":"2026-02-22T20:33:14.007350+0000","last_unstale":"2026-02-22T20:33:18.059102+0000","last_undegraded":"2026-02-22T20:33:18.059102+0000","last_fullsized":"2026-02-22T20:33:18.059102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":10,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:21.072819+0000","last_change":"2026-02-22T20:33:21.072819+0000","last_active":"2026-02-22T20:33:21.072819+0000","last_peered":"2026-02-22T20:33:14.007552+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:14.007293+0000","last_became_peered":"2026-02-22T20:33:14.007293+0000","last_unstale":"2026-02-22T20:33:21.072819+0000","last_undegraded":"2026-02-22T20:33:21.072819+0000","last_fullsized":"2026-02-22T20:33:21.072819+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":2,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:18.852969+0000","last_change":"2026-02-22T20:33:21.184030+0000","last_active":"2026-02-22T20:33:21.184030+0000","last_peered":"2026-02-22T20:33:18.852969+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:14.007287+0000","last_became_peered":"2026-02-22T20:33:14.007287+0000","last_unstale":"2026-02-22T20:33:18.852969+0000","last_undegraded":"2026-02-22T20:33:18.852969+0000","last_fullsized":"2026-02-22T20:33:18.852969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":14,"state":"peering","last_fresh":"2026-02-22T20:33:18.059137+0000","last_change":"2026-02-22T20:33:21.071028+0000","last_active":"2026-02-22T20:33:21.071028+0000","last_peered":"2026-02-22T20:33:18.059137+0000","last_clean":"2026-02-22T20:33:18.059137+0000","last_became_active":"2026-02-22T20:33:14.007683+0000","last_became_peered":"2026-02-22T20:33:14.007683+0000","last_unstale":"2026-02-22T20:33:18.059137+0000","last_undegraded":"2026-02-22T20:33:18.059137+0000","last_fullsized":"2026-02-22T20:33:18.059137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":2,"reported_epoch":15,"state":"peering","last_fresh":"2026-02-22T20:33:14.003447+0000","last_change":"2026-02-22T20:33:21.185550+0000","last_active":"2026-02-22T20:33:12.997318+0000","last_peered":"2026-02-22T20:33:12.997318+0000","last_clean":"2026-02-22T20:33:12.997318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:33:14.003447+0000","last_undegraded":"2026-02-22T20:33:14.003447+0000","last_fullsized":"2026-02-22T20:33:14.003447+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705672,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27080,"kb_used_data":120,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344760,"statfs":{"total":96636764160,"available":96609034240,"internally_reserved":0,"allocated":122880,"data_stored":34695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7550,"internal_metadata":27583106},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27080,"kb_used_data":120,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344760,"statfs":{"total":96636764160,"available":96609034240,"internally_reserved":0,"allocated":122880,"data_stored":34695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7552,"internal_metadata":27583104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705670,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705671,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:27.857 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:28.036 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:28.036 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:28.049 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2026-02-22T20:33:26.985310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54192,"kb_used_data":272,"kb_used_omap":17,"kb_used_meta":53870,"kb_avail":188689488,"statfs":{"total":193273528320,"available":193218035712,"internally_reserved":0,"allocated":278528,"data_stored":82676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17702,"internal_metadata":55163610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000964"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074340+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:22.074340+0000","last_peered":"2026-02-22T20:33:22.074340+0000","last_clean":"2026-02-22T20:33:22.074340+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:22.074340+0000","last_undegraded":"2026-02-22T20:33:22.074340+0000","last_fullsized":"2026-02-22T20:33:22.074340+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074480+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:22.074480+0000","last_peered":"2026-02-22T20:33:22.074480+0000","last_clean":"2026-02-22T20:33:22.074480+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:22.074480+0000","last_undegraded":"2026-02-22T20:33:22.074480+0000","last_fullsized":"2026-02-22T20:33:22.074480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073318+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:22.073318+0000","last_peered":"2026-02-22T20:33:22.073318+0000","last_clean":"2026-02-22T20:33:22.073318+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:22.073318+0000","last_undegraded":"2026-02-22T20:33:22.073318+0000","last_fullsized":"2026-02-22T20:33:22.073318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:22.074685+0000","last_peered":"2026-02-22T20:33:22.074685+0000","last_clean":"2026-02-22T20:33:22.074685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:22.074685+0000","last_undegraded":"2026-02-22T20:33:22.074685+0000","last_fullsized":"2026-02-22T20:33:22.074685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074257+0000","last_change":"2026-02-22T20:33:22.074638+0000","last_active":"2026-02-22T20:33:22.074257+0000","last_peered":"2026-02-22T20:33:22.074257+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:22.074257+0000","last_undegraded":"2026-02-22T20:33:22.073129+0000","last_fullsized":"2026-02-22T20:33:22.074257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074663+0000","last_change":"2026-02-22T20:33:22.074898+0000","last_active":"2026-02-22T20:33:22.074663+0000","last_peered":"2026-02-22T20:33:22.074663+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:22.074663+0000","last_undegraded":"2026-02-22T20:33:22.071487+0000","last_fullsized":"2026-02-22T20:33:22.074663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074267+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:22.074267+0000","last_peered":"2026-02-22T20:33:22.074267+0000","last_clean":"2026-02-22T20:33:22.074267+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:22.074267+0000","last_undegraded":"2026-02-22T20:33:22.074267+0000","last_fullsized":"2026-02-22T20:33:22.074267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073902+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:22.073902+0000","last_peered":"2026-02-22T20:33:22.073902+0000","last_clean":"2026-02-22T20:33:22.073902+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:22.073902+0000","last_undegraded":"2026-02-22T20:33:22.073902+0000","last_fullsized":"2026-02-22T20:33:22.073902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8850,"internal_metadata":27581806},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705672,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8852,"internal_metadata":27581804},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705671,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:28.049 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:28.227 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:28.228 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:28.239 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2026-02-22T20:33:26.985310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54192,"kb_used_data":272,"kb_used_omap":17,"kb_used_meta":53870,"kb_avail":188689488,"statfs":{"total":193273528320,"available":193218035712,"internally_reserved":0,"allocated":278528,"data_stored":82676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17702,"internal_metadata":55163610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000964"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074340+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:22.074340+0000","last_peered":"2026-02-22T20:33:22.074340+0000","last_clean":"2026-02-22T20:33:22.074340+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:22.074340+0000","last_undegraded":"2026-02-22T20:33:22.074340+0000","last_fullsized":"2026-02-22T20:33:22.074340+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074480+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:22.074480+0000","last_peered":"2026-02-22T20:33:22.074480+0000","last_clean":"2026-02-22T20:33:22.074480+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:22.074480+0000","last_undegraded":"2026-02-22T20:33:22.074480+0000","last_fullsized":"2026-02-22T20:33:22.074480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073318+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:22.073318+0000","last_peered":"2026-02-22T20:33:22.073318+0000","last_clean":"2026-02-22T20:33:22.073318+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:22.073318+0000","last_undegraded":"2026-02-22T20:33:22.073318+0000","last_fullsized":"2026-02-22T20:33:22.073318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:22.074685+0000","last_peered":"2026-02-22T20:33:22.074685+0000","last_clean":"2026-02-22T20:33:22.074685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:22.074685+0000","last_undegraded":"2026-02-22T20:33:22.074685+0000","last_fullsized":"2026-02-22T20:33:22.074685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074257+0000","last_change":"2026-02-22T20:33:22.074638+0000","last_active":"2026-02-22T20:33:22.074257+0000","last_peered":"2026-02-22T20:33:22.074257+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:22.074257+0000","last_undegraded":"2026-02-22T20:33:22.073129+0000","last_fullsized":"2026-02-22T20:33:22.074257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074663+0000","last_change":"2026-02-22T20:33:22.074898+0000","last_active":"2026-02-22T20:33:22.074663+0000","last_peered":"2026-02-22T20:33:22.074663+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:22.074663+0000","last_undegraded":"2026-02-22T20:33:22.071487+0000","last_fullsized":"2026-02-22T20:33:22.074663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074267+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:22.074267+0000","last_peered":"2026-02-22T20:33:22.074267+0000","last_clean":"2026-02-22T20:33:22.074267+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:22.074267+0000","last_undegraded":"2026-02-22T20:33:22.074267+0000","last_fullsized":"2026-02-22T20:33:22.074267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073902+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:22.073902+0000","last_peered":"2026-02-22T20:33:22.073902+0000","last_clean":"2026-02-22T20:33:22.073902+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:22.073902+0000","last_undegraded":"2026-02-22T20:33:22.073902+0000","last_fullsized":"2026-02-22T20:33:22.073902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8850,"internal_metadata":27581806},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705672,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8852,"internal_metadata":27581804},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705671,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:28.240 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:33:28.241 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 16, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-22T20:33:22.074257+0000', 'last_change': '2026-02-22T20:33:22.074638+0000', 'last_active': '2026-02-22T20:33:22.074257+0000', 'last_peered': '2026-02-22T20:33:22.074257+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:22.074257+0000', 'last_undegraded': '2026-02-22T20:33:22.073129+0000', 'last_fullsized': '2026-02-22T20:33:22.074257+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:33:28.241 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:33:28.241 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 16, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-22T20:33:22.074663+0000', 'last_change': '2026-02-22T20:33:22.074898+0000', 'last_active': '2026-02-22T20:33:22.074663+0000', 'last_peered': '2026-02-22T20:33:22.074663+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:22.074663+0000', 'last_undegraded': '2026-02-22T20:33:22.071487+0000', 'last_fullsized': '2026-02-22T20:33:22.074663+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:28.242 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:33:28.453 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:28.453 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":25,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":16,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55492608,"bytes_avail":193218035712,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-22T20:33:03.271077+0000","services":{}},"progress_events":{}} 2026-02-22T20:33:28.466 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:33:28.466 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:28.644 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:28.645 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:28.657 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2026-02-22T20:33:26.985310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54192,"kb_used_data":272,"kb_used_omap":17,"kb_used_meta":53870,"kb_avail":188689488,"statfs":{"total":193273528320,"available":193218035712,"internally_reserved":0,"allocated":278528,"data_stored":82676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17702,"internal_metadata":55163610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000964"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074340+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:22.074340+0000","last_peered":"2026-02-22T20:33:22.074340+0000","last_clean":"2026-02-22T20:33:22.074340+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:22.074340+0000","last_undegraded":"2026-02-22T20:33:22.074340+0000","last_fullsized":"2026-02-22T20:33:22.074340+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074480+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:22.074480+0000","last_peered":"2026-02-22T20:33:22.074480+0000","last_clean":"2026-02-22T20:33:22.074480+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:22.074480+0000","last_undegraded":"2026-02-22T20:33:22.074480+0000","last_fullsized":"2026-02-22T20:33:22.074480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073318+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:22.073318+0000","last_peered":"2026-02-22T20:33:22.073318+0000","last_clean":"2026-02-22T20:33:22.073318+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:22.073318+0000","last_undegraded":"2026-02-22T20:33:22.073318+0000","last_fullsized":"2026-02-22T20:33:22.073318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:22.074685+0000","last_peered":"2026-02-22T20:33:22.074685+0000","last_clean":"2026-02-22T20:33:22.074685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:22.074685+0000","last_undegraded":"2026-02-22T20:33:22.074685+0000","last_fullsized":"2026-02-22T20:33:22.074685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074257+0000","last_change":"2026-02-22T20:33:22.074638+0000","last_active":"2026-02-22T20:33:22.074257+0000","last_peered":"2026-02-22T20:33:22.074257+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:22.074257+0000","last_undegraded":"2026-02-22T20:33:22.073129+0000","last_fullsized":"2026-02-22T20:33:22.074257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074663+0000","last_change":"2026-02-22T20:33:22.074898+0000","last_active":"2026-02-22T20:33:22.074663+0000","last_peered":"2026-02-22T20:33:22.074663+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:22.074663+0000","last_undegraded":"2026-02-22T20:33:22.071487+0000","last_fullsized":"2026-02-22T20:33:22.074663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074267+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:22.074267+0000","last_peered":"2026-02-22T20:33:22.074267+0000","last_clean":"2026-02-22T20:33:22.074267+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:22.074267+0000","last_undegraded":"2026-02-22T20:33:22.074267+0000","last_fullsized":"2026-02-22T20:33:22.074267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073902+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:22.073902+0000","last_peered":"2026-02-22T20:33:22.073902+0000","last_clean":"2026-02-22T20:33:22.073902+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:22.073902+0000","last_undegraded":"2026-02-22T20:33:22.073902+0000","last_fullsized":"2026-02-22T20:33:22.073902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8850,"internal_metadata":27581806},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705672,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8852,"internal_metadata":27581804},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705671,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:31.659 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:31.839 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:31.839 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:31.851 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2026-02-22T20:33:30.985860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54192,"kb_used_data":272,"kb_used_omap":17,"kb_used_meta":53870,"kb_avail":188689488,"statfs":{"total":193273528320,"available":193218035712,"internally_reserved":0,"allocated":278528,"data_stored":82676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17702,"internal_metadata":55163610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.916613"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074340+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:22.074340+0000","last_peered":"2026-02-22T20:33:22.074340+0000","last_clean":"2026-02-22T20:33:22.074340+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:22.074340+0000","last_undegraded":"2026-02-22T20:33:22.074340+0000","last_fullsized":"2026-02-22T20:33:22.074340+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074480+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:22.074480+0000","last_peered":"2026-02-22T20:33:22.074480+0000","last_clean":"2026-02-22T20:33:22.074480+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:22.074480+0000","last_undegraded":"2026-02-22T20:33:22.074480+0000","last_fullsized":"2026-02-22T20:33:22.074480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073318+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:22.073318+0000","last_peered":"2026-02-22T20:33:22.073318+0000","last_clean":"2026-02-22T20:33:22.073318+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:22.073318+0000","last_undegraded":"2026-02-22T20:33:22.073318+0000","last_fullsized":"2026-02-22T20:33:22.073318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:22.074685+0000","last_peered":"2026-02-22T20:33:22.074685+0000","last_clean":"2026-02-22T20:33:22.074685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:22.074685+0000","last_undegraded":"2026-02-22T20:33:22.074685+0000","last_fullsized":"2026-02-22T20:33:22.074685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074257+0000","last_change":"2026-02-22T20:33:22.074638+0000","last_active":"2026-02-22T20:33:22.074257+0000","last_peered":"2026-02-22T20:33:22.074257+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:22.074257+0000","last_undegraded":"2026-02-22T20:33:22.073129+0000","last_fullsized":"2026-02-22T20:33:22.074257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074663+0000","last_change":"2026-02-22T20:33:22.074898+0000","last_active":"2026-02-22T20:33:22.074663+0000","last_peered":"2026-02-22T20:33:22.074663+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:22.074663+0000","last_undegraded":"2026-02-22T20:33:22.071487+0000","last_fullsized":"2026-02-22T20:33:22.074663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074267+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:22.074267+0000","last_peered":"2026-02-22T20:33:22.074267+0000","last_clean":"2026-02-22T20:33:22.074267+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:22.074267+0000","last_undegraded":"2026-02-22T20:33:22.074267+0000","last_fullsized":"2026-02-22T20:33:22.074267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073902+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:22.073902+0000","last_peered":"2026-02-22T20:33:22.073902+0000","last_clean":"2026-02-22T20:33:22.073902+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:22.073902+0000","last_undegraded":"2026-02-22T20:33:22.073902+0000","last_fullsized":"2026-02-22T20:33:22.073902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8850,"internal_metadata":27581806},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8852,"internal_metadata":27581804},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:31.852 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:32.029 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:32.030 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:32.042 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2026-02-22T20:33:30.985860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54192,"kb_used_data":272,"kb_used_omap":17,"kb_used_meta":53870,"kb_avail":188689488,"statfs":{"total":193273528320,"available":193218035712,"internally_reserved":0,"allocated":278528,"data_stored":82676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17702,"internal_metadata":55163610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.916613"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074340+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:22.074340+0000","last_peered":"2026-02-22T20:33:22.074340+0000","last_clean":"2026-02-22T20:33:22.074340+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:22.074340+0000","last_undegraded":"2026-02-22T20:33:22.074340+0000","last_fullsized":"2026-02-22T20:33:22.074340+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074480+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:22.074480+0000","last_peered":"2026-02-22T20:33:22.074480+0000","last_clean":"2026-02-22T20:33:22.074480+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:22.074480+0000","last_undegraded":"2026-02-22T20:33:22.074480+0000","last_fullsized":"2026-02-22T20:33:22.074480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073318+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:22.073318+0000","last_peered":"2026-02-22T20:33:22.073318+0000","last_clean":"2026-02-22T20:33:22.073318+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:22.073318+0000","last_undegraded":"2026-02-22T20:33:22.073318+0000","last_fullsized":"2026-02-22T20:33:22.073318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:22.074685+0000","last_peered":"2026-02-22T20:33:22.074685+0000","last_clean":"2026-02-22T20:33:22.074685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:22.074685+0000","last_undegraded":"2026-02-22T20:33:22.074685+0000","last_fullsized":"2026-02-22T20:33:22.074685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074257+0000","last_change":"2026-02-22T20:33:22.074638+0000","last_active":"2026-02-22T20:33:22.074257+0000","last_peered":"2026-02-22T20:33:22.074257+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:22.074257+0000","last_undegraded":"2026-02-22T20:33:22.073129+0000","last_fullsized":"2026-02-22T20:33:22.074257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074663+0000","last_change":"2026-02-22T20:33:22.074898+0000","last_active":"2026-02-22T20:33:22.074663+0000","last_peered":"2026-02-22T20:33:22.074663+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:22.074663+0000","last_undegraded":"2026-02-22T20:33:22.071487+0000","last_fullsized":"2026-02-22T20:33:22.074663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074267+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:22.074267+0000","last_peered":"2026-02-22T20:33:22.074267+0000","last_clean":"2026-02-22T20:33:22.074267+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:22.074267+0000","last_undegraded":"2026-02-22T20:33:22.074267+0000","last_fullsized":"2026-02-22T20:33:22.074267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073902+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:22.073902+0000","last_peered":"2026-02-22T20:33:22.073902+0000","last_clean":"2026-02-22T20:33:22.073902+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:22.073902+0000","last_undegraded":"2026-02-22T20:33:22.073902+0000","last_fullsized":"2026-02-22T20:33:22.073902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8850,"internal_metadata":27581806},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8852,"internal_metadata":27581804},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:32.043 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:33:32.043 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 16, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-22T20:33:22.074257+0000', 'last_change': '2026-02-22T20:33:22.074638+0000', 'last_active': '2026-02-22T20:33:22.074257+0000', 'last_peered': '2026-02-22T20:33:22.074257+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:22.074257+0000', 'last_undegraded': '2026-02-22T20:33:22.073129+0000', 'last_fullsized': '2026-02-22T20:33:22.074257+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:33:32.044 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:33:32.044 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 16, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-22T20:33:22.074663+0000', 'last_change': '2026-02-22T20:33:22.074898+0000', 'last_active': '2026-02-22T20:33:22.074663+0000', 'last_peered': '2026-02-22T20:33:22.074663+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:22.074663+0000', 'last_undegraded': '2026-02-22T20:33:22.071487+0000', 'last_fullsized': '2026-02-22T20:33:22.074663+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:32.044 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:33:32.260 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:32.260 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":16,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55492608,"bytes_avail":193218035712,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-22T20:33:03.271077+0000","services":{}},"progress_events":{}} 2026-02-22T20:33:32.274 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:33:32.274 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:32.452 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:32.453 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:32.465 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2026-02-22T20:33:30.985860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54192,"kb_used_data":272,"kb_used_omap":17,"kb_used_meta":53870,"kb_avail":188689488,"statfs":{"total":193273528320,"available":193218035712,"internally_reserved":0,"allocated":278528,"data_stored":82676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17702,"internal_metadata":55163610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.916613"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074340+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:22.074340+0000","last_peered":"2026-02-22T20:33:22.074340+0000","last_clean":"2026-02-22T20:33:22.074340+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:22.074340+0000","last_undegraded":"2026-02-22T20:33:22.074340+0000","last_fullsized":"2026-02-22T20:33:22.074340+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074480+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:22.074480+0000","last_peered":"2026-02-22T20:33:22.074480+0000","last_clean":"2026-02-22T20:33:22.074480+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:22.074480+0000","last_undegraded":"2026-02-22T20:33:22.074480+0000","last_fullsized":"2026-02-22T20:33:22.074480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073318+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:22.073318+0000","last_peered":"2026-02-22T20:33:22.073318+0000","last_clean":"2026-02-22T20:33:22.073318+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:22.073318+0000","last_undegraded":"2026-02-22T20:33:22.073318+0000","last_fullsized":"2026-02-22T20:33:22.073318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:22.074685+0000","last_peered":"2026-02-22T20:33:22.074685+0000","last_clean":"2026-02-22T20:33:22.074685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:22.074685+0000","last_undegraded":"2026-02-22T20:33:22.074685+0000","last_fullsized":"2026-02-22T20:33:22.074685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074257+0000","last_change":"2026-02-22T20:33:22.074638+0000","last_active":"2026-02-22T20:33:22.074257+0000","last_peered":"2026-02-22T20:33:22.074257+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:22.074257+0000","last_undegraded":"2026-02-22T20:33:22.073129+0000","last_fullsized":"2026-02-22T20:33:22.074257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074663+0000","last_change":"2026-02-22T20:33:22.074898+0000","last_active":"2026-02-22T20:33:22.074663+0000","last_peered":"2026-02-22T20:33:22.074663+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:22.074663+0000","last_undegraded":"2026-02-22T20:33:22.071487+0000","last_fullsized":"2026-02-22T20:33:22.074663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074267+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:22.074267+0000","last_peered":"2026-02-22T20:33:22.074267+0000","last_clean":"2026-02-22T20:33:22.074267+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:22.074267+0000","last_undegraded":"2026-02-22T20:33:22.074267+0000","last_fullsized":"2026-02-22T20:33:22.074267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073902+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:22.073902+0000","last_peered":"2026-02-22T20:33:22.073902+0000","last_clean":"2026-02-22T20:33:22.073902+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:22.073902+0000","last_undegraded":"2026-02-22T20:33:22.073902+0000","last_fullsized":"2026-02-22T20:33:22.073902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8850,"internal_metadata":27581806},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8852,"internal_metadata":27581804},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:35.467 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:35.646 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:35.646 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:35.658 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2026-02-22T20:33:34.986380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54192,"kb_used_data":272,"kb_used_omap":17,"kb_used_meta":53870,"kb_avail":188689488,"statfs":{"total":193273528320,"available":193218035712,"internally_reserved":0,"allocated":278528,"data_stored":82676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17702,"internal_metadata":55163610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001270"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074340+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:22.074340+0000","last_peered":"2026-02-22T20:33:22.074340+0000","last_clean":"2026-02-22T20:33:22.074340+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:22.074340+0000","last_undegraded":"2026-02-22T20:33:22.074340+0000","last_fullsized":"2026-02-22T20:33:22.074340+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074480+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:22.074480+0000","last_peered":"2026-02-22T20:33:22.074480+0000","last_clean":"2026-02-22T20:33:22.074480+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:22.074480+0000","last_undegraded":"2026-02-22T20:33:22.074480+0000","last_fullsized":"2026-02-22T20:33:22.074480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073318+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:22.073318+0000","last_peered":"2026-02-22T20:33:22.073318+0000","last_clean":"2026-02-22T20:33:22.073318+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:22.073318+0000","last_undegraded":"2026-02-22T20:33:22.073318+0000","last_fullsized":"2026-02-22T20:33:22.073318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:22.074685+0000","last_peered":"2026-02-22T20:33:22.074685+0000","last_clean":"2026-02-22T20:33:22.074685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:22.074685+0000","last_undegraded":"2026-02-22T20:33:22.074685+0000","last_fullsized":"2026-02-22T20:33:22.074685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074257+0000","last_change":"2026-02-22T20:33:22.074638+0000","last_active":"2026-02-22T20:33:22.074257+0000","last_peered":"2026-02-22T20:33:22.074257+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:22.074257+0000","last_undegraded":"2026-02-22T20:33:22.073129+0000","last_fullsized":"2026-02-22T20:33:22.074257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074663+0000","last_change":"2026-02-22T20:33:22.074898+0000","last_active":"2026-02-22T20:33:22.074663+0000","last_peered":"2026-02-22T20:33:22.074663+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:22.074663+0000","last_undegraded":"2026-02-22T20:33:22.071487+0000","last_fullsized":"2026-02-22T20:33:22.074663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074267+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:22.074267+0000","last_peered":"2026-02-22T20:33:22.074267+0000","last_clean":"2026-02-22T20:33:22.074267+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:22.074267+0000","last_undegraded":"2026-02-22T20:33:22.074267+0000","last_fullsized":"2026-02-22T20:33:22.074267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073902+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:22.073902+0000","last_peered":"2026-02-22T20:33:22.073902+0000","last_clean":"2026-02-22T20:33:22.073902+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:22.073902+0000","last_undegraded":"2026-02-22T20:33:22.073902+0000","last_fullsized":"2026-02-22T20:33:22.073902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8850,"internal_metadata":27581806},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8852,"internal_metadata":27581804},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:35.659 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:35.832 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:35.832 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:35.845 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2026-02-22T20:33:34.986380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54192,"kb_used_data":272,"kb_used_omap":17,"kb_used_meta":53870,"kb_avail":188689488,"statfs":{"total":193273528320,"available":193218035712,"internally_reserved":0,"allocated":278528,"data_stored":82676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17702,"internal_metadata":55163610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001270"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074340+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:22.074340+0000","last_peered":"2026-02-22T20:33:22.074340+0000","last_clean":"2026-02-22T20:33:22.074340+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:22.074340+0000","last_undegraded":"2026-02-22T20:33:22.074340+0000","last_fullsized":"2026-02-22T20:33:22.074340+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074480+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:22.074480+0000","last_peered":"2026-02-22T20:33:22.074480+0000","last_clean":"2026-02-22T20:33:22.074480+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:22.074480+0000","last_undegraded":"2026-02-22T20:33:22.074480+0000","last_fullsized":"2026-02-22T20:33:22.074480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073318+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:22.073318+0000","last_peered":"2026-02-22T20:33:22.073318+0000","last_clean":"2026-02-22T20:33:22.073318+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:22.073318+0000","last_undegraded":"2026-02-22T20:33:22.073318+0000","last_fullsized":"2026-02-22T20:33:22.073318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:22.074685+0000","last_peered":"2026-02-22T20:33:22.074685+0000","last_clean":"2026-02-22T20:33:22.074685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:22.074685+0000","last_undegraded":"2026-02-22T20:33:22.074685+0000","last_fullsized":"2026-02-22T20:33:22.074685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074257+0000","last_change":"2026-02-22T20:33:22.074638+0000","last_active":"2026-02-22T20:33:22.074257+0000","last_peered":"2026-02-22T20:33:22.074257+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:22.074257+0000","last_undegraded":"2026-02-22T20:33:22.073129+0000","last_fullsized":"2026-02-22T20:33:22.074257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074663+0000","last_change":"2026-02-22T20:33:22.074898+0000","last_active":"2026-02-22T20:33:22.074663+0000","last_peered":"2026-02-22T20:33:22.074663+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:22.074663+0000","last_undegraded":"2026-02-22T20:33:22.071487+0000","last_fullsized":"2026-02-22T20:33:22.074663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074267+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:22.074267+0000","last_peered":"2026-02-22T20:33:22.074267+0000","last_clean":"2026-02-22T20:33:22.074267+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:22.074267+0000","last_undegraded":"2026-02-22T20:33:22.074267+0000","last_fullsized":"2026-02-22T20:33:22.074267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073902+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:22.073902+0000","last_peered":"2026-02-22T20:33:22.073902+0000","last_clean":"2026-02-22T20:33:22.073902+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:22.073902+0000","last_undegraded":"2026-02-22T20:33:22.073902+0000","last_fullsized":"2026-02-22T20:33:22.073902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8850,"internal_metadata":27581806},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8852,"internal_metadata":27581804},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:35.846 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:33:35.846 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 17, 'reported_epoch': 16, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-22T20:33:22.074257+0000', 'last_change': '2026-02-22T20:33:22.074638+0000', 'last_active': '2026-02-22T20:33:22.074257+0000', 'last_peered': '2026-02-22T20:33:22.074257+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:22.074257+0000', 'last_undegraded': '2026-02-22T20:33:22.073129+0000', 'last_fullsized': '2026-02-22T20:33:22.074257+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:33:35.846 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:33:35.846 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 10, 'reported_epoch': 16, 'state': 'active+recovering+degraded', 'last_fresh': '2026-02-22T20:33:22.074663+0000', 'last_change': '2026-02-22T20:33:22.074898+0000', 'last_active': '2026-02-22T20:33:22.074663+0000', 'last_peered': '2026-02-22T20:33:22.074663+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:22.074663+0000', 'last_undegraded': '2026-02-22T20:33:22.071487+0000', 'last_fullsized': '2026-02-22T20:33:22.074663+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:35.847 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:33:36.061 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:36.061 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":32,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55492608,"bytes_avail":193218035712,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-22T20:33:03.271077+0000","services":{}},"progress_events":{}} 2026-02-22T20:33:36.073 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:33:36.073 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:36.251 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:36.251 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:36.263 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2026-02-22T20:33:34.986380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54192,"kb_used_data":272,"kb_used_omap":17,"kb_used_meta":53870,"kb_avail":188689488,"statfs":{"total":193273528320,"available":193218035712,"internally_reserved":0,"allocated":278528,"data_stored":82676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17702,"internal_metadata":55163610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":1,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001270"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074340+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:22.074340+0000","last_peered":"2026-02-22T20:33:22.074340+0000","last_clean":"2026-02-22T20:33:22.074340+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:22.074340+0000","last_undegraded":"2026-02-22T20:33:22.074340+0000","last_fullsized":"2026-02-22T20:33:22.074340+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074480+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:22.074480+0000","last_peered":"2026-02-22T20:33:22.074480+0000","last_clean":"2026-02-22T20:33:22.074480+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:22.074480+0000","last_undegraded":"2026-02-22T20:33:22.074480+0000","last_fullsized":"2026-02-22T20:33:22.074480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073318+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:22.073318+0000","last_peered":"2026-02-22T20:33:22.073318+0000","last_clean":"2026-02-22T20:33:22.073318+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:22.073318+0000","last_undegraded":"2026-02-22T20:33:22.073318+0000","last_fullsized":"2026-02-22T20:33:22.073318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:22.074685+0000","last_peered":"2026-02-22T20:33:22.074685+0000","last_clean":"2026-02-22T20:33:22.074685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:22.074685+0000","last_undegraded":"2026-02-22T20:33:22.074685+0000","last_fullsized":"2026-02-22T20:33:22.074685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":17,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074257+0000","last_change":"2026-02-22T20:33:22.074638+0000","last_active":"2026-02-22T20:33:22.074257+0000","last_peered":"2026-02-22T20:33:22.074257+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:22.074257+0000","last_undegraded":"2026-02-22T20:33:22.073129+0000","last_fullsized":"2026-02-22T20:33:22.074257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":10,"reported_epoch":16,"state":"active+recovering+degraded","last_fresh":"2026-02-22T20:33:22.074663+0000","last_change":"2026-02-22T20:33:22.074898+0000","last_active":"2026-02-22T20:33:22.074663+0000","last_peered":"2026-02-22T20:33:22.074663+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:22.074663+0000","last_undegraded":"2026-02-22T20:33:22.071487+0000","last_fullsized":"2026-02-22T20:33:22.074663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.074267+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:22.074267+0000","last_peered":"2026-02-22T20:33:22.074267+0000","last_clean":"2026-02-22T20:33:22.074267+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:22.074267+0000","last_undegraded":"2026-02-22T20:33:22.074267+0000","last_fullsized":"2026-02-22T20:33:22.074267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-22T20:33:22.073902+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:22.073902+0000","last_peered":"2026-02-22T20:33:22.073902+0000","last_clean":"2026-02-22T20:33:22.073902+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:22.073902+0000","last_undegraded":"2026-02-22T20:33:22.073902+0000","last_fullsized":"2026-02-22T20:33:22.073902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8850,"internal_metadata":27581806},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":41338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8852,"internal_metadata":27581804},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705672,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:39.265 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:39.445 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:39.445 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:39.457 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2026-02-22T20:33:38.986908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001345"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:39.458 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:39.637 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:39.637 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:39.649 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2026-02-22T20:33:38.986908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001345"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:39.650 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:33:39.650 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:33:39.650 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:33:39.650 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:39.650 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:33:39.862 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:39.863 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":36,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-22T20:33:03.271077+0000","services":{}},"progress_events":{}} 2026-02-22T20:33:39.875 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:33:39.876 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:40.057 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:40.057 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:40.070 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2026-02-22T20:33:38.986908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001345"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705673,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:43.071 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:43.248 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:43.249 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:43.261 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2026-02-22T20:33:42.987443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001340"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:43.263 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:43.441 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:43.442 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:43.454 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2026-02-22T20:33:42.987443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001340"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:43.456 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:33:43.457 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:33:43.457 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:33:43.457 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:43.458 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:33:43.676 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:43.676 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":40,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-22T20:33:03.271077+0000","services":{}},"progress_events":{}} 2026-02-22T20:33:43.689 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:33:43.689 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:43.868 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:43.868 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:43.880 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2026-02-22T20:33:42.987443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001340"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705675,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705674,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:46.882 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:47.062 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:47.062 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:47.074 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2026-02-22T20:33:46.987955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001574"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:47.075 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:47.257 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:47.258 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:47.270 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2026-02-22T20:33:46.987955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001574"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:47.272 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:33:47.273 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:33:47.273 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:33:47.273 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:47.274 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:33:47.489 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:47.489 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":44,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-22T20:33:03.271077+0000","services":{}},"progress_events":{}} 2026-02-22T20:33:47.502 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:33:47.502 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:47.678 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:47.679 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:47.690 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2026-02-22T20:33:46.987955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001574"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:50.693 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:50.874 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:50.875 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:50.887 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2026-02-22T20:33:48.988179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001527"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705676,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705675,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:50.889 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:51.068 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:51.068 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:51.080 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2026-02-22T20:33:50.988432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001523"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:51.081 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:33:51.081 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:33:51.081 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:33:51.081 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:51.081 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:33:51.296 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:51.296 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":48,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-22T20:33:03.271077+0000","services":{}},"progress_events":{}} 2026-02-22T20:33:51.309 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:33:51.309 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:51.487 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:51.488 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:51.499 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2026-02-22T20:33:50.988432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001523"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:54.501 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:54.684 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:54.685 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:54.697 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2026-02-22T20:33:52.988669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001525"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:54.698 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:54.875 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:54.876 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:54.887 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2026-02-22T20:33:52.988669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001525"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:54.889 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:33:54.889 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:33:54.889 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:33:54.889 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:54.890 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:33:55.104 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:55.105 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":51,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-22T20:33:03.271077+0000","services":{}},"progress_events":{}} 2026-02-22T20:33:55.117 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:33:55.118 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:55.295 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:55.296 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:55.308 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2026-02-22T20:33:54.988927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001483"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705677,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705676,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:58.310 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:58.487 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:58.488 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:58.499 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2026-02-22T20:33:56.989151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001483"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:58.500 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:58.679 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:58.680 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:58.692 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2026-02-22T20:33:56.989151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001483"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:33:58.694 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:33:58.694 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:33:58.695 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:33:58.695 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:33:58.695 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:33:58.912 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:58.912 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":55,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-22T20:33:03.271077+0000","services":{}},"progress_events":{}} 2026-02-22T20:33:58.925 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:33:58.926 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:33:59.109 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:33:59.110 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:33:59.121 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2026-02-22T20:33:58.989447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001491"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705678,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705677,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:02.123 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:02.302 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:02.303 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:02.314 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2026-02-22T20:34:00.989672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:02.316 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:02.497 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:02.498 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:02.509 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2026-02-22T20:34:00.989672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:02.511 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:02.511 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:02.511 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:02.511 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:02.512 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:02.726 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:02.726 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":59,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-22T20:33:03.271077+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:02.739 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:02.739 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:02.917 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:02.917 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:02.929 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2026-02-22T20:34:00.989672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:05.931 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:06.111 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:06.111 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:06.122 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2026-02-22T20:34:04.990158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001488"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:06.124 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:06.300 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:06.301 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:06.313 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2026-02-22T20:34:04.990158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001488"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:06.314 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:06.314 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:06.314 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:06.315 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:06.315 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:06.531 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:06.531 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":63,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":1,"modified":"2026-02-22T20:33:03.271077+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:06.544 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:06.545 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:06.724 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:06.725 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:06.737 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2026-02-22T20:34:04.990158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001488"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705679,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705678,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:09.739 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:09.922 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:09.922 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:09.934 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2026-02-22T20:34:08.990636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001484"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:09.936 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:10.114 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:10.114 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:10.127 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2026-02-22T20:34:08.990636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001484"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:10.128 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:10.128 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:10.128 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:10.128 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:10.129 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:10.344 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:10.344 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":67,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":2,"modified":"2026-02-22T20:34:06.990406+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2026-02-22T20:34:10.358 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:10.358 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:10.536 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:10.536 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:10.547 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2026-02-22T20:34:08.990636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001484"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705680,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705679,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:13.550 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:13.729 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:13.729 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:13.741 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2026-02-22T20:34:12.991164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001491"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:13.743 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:13.921 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:13.921 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:13.934 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2026-02-22T20:34:12.991164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001491"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:13.935 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:13.935 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:13.936 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:13.936 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:13.936 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:14.153 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:14.154 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":70,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:14.166 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:14.167 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:14.343 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:14.343 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:14.354 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2026-02-22T20:34:12.991164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001491"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705681,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705680,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:17.355 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:17.536 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:17.536 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:17.547 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2026-02-22T20:34:16.991650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:17.549 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:17.729 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:17.730 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:17.741 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2026-02-22T20:34:16.991650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:17.742 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:17.742 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:17.742 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:17.742 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:17.742 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:17.956 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:17.957 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":74,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:17.970 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:17.970 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:18.149 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:18.150 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:18.161 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2026-02-22T20:34:16.991650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705682,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705681,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:21.164 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:21.343 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:21.344 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:21.356 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2026-02-22T20:34:20.992172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001536"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:21.357 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:21.533 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:21.534 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:21.546 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2026-02-22T20:34:20.992172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001536"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:21.547 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:21.548 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:21.548 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:21.548 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:21.549 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:21.762 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:21.762 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":78,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:21.774 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:21.774 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:21.951 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:21.951 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:21.963 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2026-02-22T20:34:20.992172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001536"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:24.965 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:25.145 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:25.145 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:25.158 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2026-02-22T20:34:24.992622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001458"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:25.159 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:25.337 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:25.338 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:25.349 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2026-02-22T20:34:24.992622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001458"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:25.352 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:25.352 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:25.352 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:25.353 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:25.353 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:25.566 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:25.566 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":82,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:25.578 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:25.578 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:25.755 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:25.755 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:25.767 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2026-02-22T20:34:24.992622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001458"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705682,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:28.768 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:28.948 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:28.948 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:28.959 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2026-02-22T20:34:26.992814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001440"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:28.960 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:29.140 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:29.141 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:29.153 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2026-02-22T20:34:28.993113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001462"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:29.154 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:29.154 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:29.154 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:29.154 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:29.155 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:29.372 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:29.373 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":86,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:29.387 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:29.387 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:29.565 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:29.566 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:29.578 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2026-02-22T20:34:28.993113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001462"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705684,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705683,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:32.580 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:32.762 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:32.763 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:32.775 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2026-02-22T20:34:30.993311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001407"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:32.776 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:32.959 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:32.960 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:32.972 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2026-02-22T20:34:30.993311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001407"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:32.973 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:32.973 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:32.974 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:32.974 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:32.974 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:33.190 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:33.190 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":89,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:33.202 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:33.203 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:33.386 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:33.387 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:33.398 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2026-02-22T20:34:32.993501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001328"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:36.401 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:36.579 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:36.580 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:36.591 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2026-02-22T20:34:34.993741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001355"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:36.593 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:36.775 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:36.776 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:36.788 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2026-02-22T20:34:34.993741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001355"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705684,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:36.789 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:36.789 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:36.790 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:36.790 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:36.790 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:37.005 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:37.006 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":93,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:37.019 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:37.020 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:37.199 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:37.199 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:37.211 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2026-02-22T20:34:36.993993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001370"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:40.213 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:40.393 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:40.394 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:40.406 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2026-02-22T20:34:38.994229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001414"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:40.407 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:40.587 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:40.587 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:40.600 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2026-02-22T20:34:38.994229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001414"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705686,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705685,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:40.601 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:40.601 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:40.601 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:40.601 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:40.602 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:40.817 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:40.817 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":97,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:40.830 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:40.830 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:41.007 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:41.007 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:41.019 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2026-02-22T20:34:40.994496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001382"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:44.021 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:44.203 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:44.204 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:44.215 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2026-02-22T20:34:42.994727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001416"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:44.217 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:44.393 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:44.393 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:44.405 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2026-02-22T20:34:42.994727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001416"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:44.406 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:44.406 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:44.407 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:44.407 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:44.407 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:44.640 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:44.640 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":101,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:44.652 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:44.653 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:44.831 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:44.831 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:44.844 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2026-02-22T20:34:42.994727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001416"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705687,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705686,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:47.846 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:48.024 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:48.025 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:48.036 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2026-02-22T20:34:46.995290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001548"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:48.037 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:48.214 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:48.215 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:48.226 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2026-02-22T20:34:46.995290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001548"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:48.227 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:48.228 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:48.228 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:48.228 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:48.228 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:48.447 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:48.447 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":105,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:48.460 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:48.460 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:48.635 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:48.636 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:48.647 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2026-02-22T20:34:46.995290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001548"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705688,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705687,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:51.649 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:51.828 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:51.828 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:51.840 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2026-02-22T20:34:50.995782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001553"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:51.842 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:52.019 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:52.020 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:52.031 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2026-02-22T20:34:50.995782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001553"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:52.033 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:52.033 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:52.033 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:52.033 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:52.034 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:52.246 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:52.246 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":108,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:52.258 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:52.258 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:52.436 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:52.436 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:52.448 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2026-02-22T20:34:50.995782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001553"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:55.450 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:55.628 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:55.628 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:55.640 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2026-02-22T20:34:54.996295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001567"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:55.642 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:55.820 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:55.821 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:55.833 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2026-02-22T20:34:54.996295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001567"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:55.835 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:55.836 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:55.836 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:55.836 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:55.837 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:56.057 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:56.057 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":112,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:56.069 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:56.070 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:56.245 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:56.245 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:56.256 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2026-02-22T20:34:54.996295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001567"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705689,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705688,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:59.257 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:59.435 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:59.436 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:59.448 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2026-02-22T20:34:58.996817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001527"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:59.450 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:34:59.625 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:59.626 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:34:59.638 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2026-02-22T20:34:58.996817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001527"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:34:59.639 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:34:59.640 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:34:59.640 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:34:59.640 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:34:59.640 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:34:59.859 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:34:59.860 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":116,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:34:59.872 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:34:59.872 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:00.051 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:00.051 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:00.063 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2026-02-22T20:34:58.996817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001527"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705690,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705689,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:03.065 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:03.244 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:03.245 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:03.256 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2026-02-22T20:35:02.997228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001446"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:03.257 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:03.438 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:03.439 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:03.451 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2026-02-22T20:35:02.997228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001446"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:03.453 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:03.453 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:03.453 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:03.453 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:03.454 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:03.673 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:03.674 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":120,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:03.686 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:03.686 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:03.866 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:03.866 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:03.879 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2026-02-22T20:35:02.997228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001446"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705690,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:06.881 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:07.060 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:07.061 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:07.072 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2026-02-22T20:35:06.997647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001352"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:07.073 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:07.251 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:07.252 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:07.263 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2026-02-22T20:35:06.997647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001352"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:07.264 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:07.264 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:07.265 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:07.265 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:07.265 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:07.478 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:07.478 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":124,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:07.491 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:07.491 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:07.666 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:07.666 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:07.677 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2026-02-22T20:35:06.997647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001352"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:10.678 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:10.859 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:10.859 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:10.871 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2026-02-22T20:35:08.997888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001366"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705692,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705691,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:10.872 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:11.060 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:11.060 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:11.073 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2026-02-22T20:35:10.998150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001332"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:11.073 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:11.073 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:11.073 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:11.073 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:11.074 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:11.286 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:11.287 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":128,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:11.300 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:11.301 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:11.481 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:11.482 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:11.493 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2026-02-22T20:35:10.998150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001332"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:14.495 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:14.674 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:14.675 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:14.685 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2026-02-22T20:35:12.998395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001383"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:14.686 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:14.864 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:14.865 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:14.876 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2026-02-22T20:35:12.998395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001383"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:14.876 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:14.876 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:14.877 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:14.877 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:14.877 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:15.088 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:15.088 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":131,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:15.101 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:15.101 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:15.280 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:15.280 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:15.293 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2026-02-22T20:35:14.998669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001441"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705693,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705692,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:18.294 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:18.471 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:18.471 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:18.483 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2026-02-22T20:35:16.998907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001477"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:18.484 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:18.661 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:18.661 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:18.673 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2026-02-22T20:35:16.998907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001477"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:18.674 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:18.674 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:18.675 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:18.675 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:18.675 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:18.891 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:18.891 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":135,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:18.903 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:18.904 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:19.081 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:19.081 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:19.093 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2026-02-22T20:35:18.999159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001512"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143506+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143506+0000","last_peered":"2026-02-22T20:33:33.143506+0000","last_clean":"2026-02-22T20:33:33.143506+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143506+0000","last_undegraded":"2026-02-22T20:33:33.143506+0000","last_fullsized":"2026-02-22T20:33:33.143506+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143172+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143172+0000","last_peered":"2026-02-22T20:33:33.143172+0000","last_clean":"2026-02-22T20:33:33.143172+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143172+0000","last_undegraded":"2026-02-22T20:33:33.143172+0000","last_fullsized":"2026-02-22T20:33:33.143172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143058+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143058+0000","last_peered":"2026-02-22T20:33:33.143058+0000","last_clean":"2026-02-22T20:33:33.143058+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143058+0000","last_undegraded":"2026-02-22T20:33:33.143058+0000","last_fullsized":"2026-02-22T20:33:33.143058+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705693,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:22.095 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:22.280 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:22.281 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:22.293 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2026-02-22T20:35:20.999353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001464"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:22.295 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:22.471 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:22.472 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:22.483 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2026-02-22T20:35:20.999353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001464"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:22.485 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:22.485 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:22.485 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:22.485 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:22.486 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:22.699 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:22.699 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":139,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:22.711 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:22.712 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:22.890 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:22.890 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:22.902 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2026-02-22T20:35:20.999353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001464"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:25.905 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:26.086 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:26.086 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:26.098 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2026-02-22T20:35:24.999915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001520"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:26.099 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:26.278 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:26.278 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:26.290 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2026-02-22T20:35:24.999915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001520"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:26.291 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:26.292 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143372+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143372+0000', 'last_peered': '2026-02-22T20:33:33.143372+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143372+0000', 'last_undegraded': '2026-02-22T20:33:33.143372+0000', 'last_fullsized': '2026-02-22T20:33:33.143372+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:26.292 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:26.292 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 12, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143053+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143053+0000', 'last_peered': '2026-02-22T20:33:33.143053+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143053+0000', 'last_undegraded': '2026-02-22T20:33:33.143053+0000', 'last_fullsized': '2026-02-22T20:33:33.143053+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:26.292 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:26.513 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:26.513 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":143,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:26.527 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:26.528 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:26.707 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:26.708 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:26.720 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2026-02-22T20:35:24.999915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001520"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143218+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143218+0000","last_peered":"2026-02-22T20:33:33.143218+0000","last_clean":"2026-02-22T20:33:33.143218+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143218+0000","last_undegraded":"2026-02-22T20:33:33.143218+0000","last_fullsized":"2026-02-22T20:33:33.143218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143372+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143372+0000","last_peered":"2026-02-22T20:33:33.143372+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143372+0000","last_undegraded":"2026-02-22T20:33:33.143372+0000","last_fullsized":"2026-02-22T20:33:33.143372+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":12,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143053+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143053+0000","last_peered":"2026-02-22T20:33:33.143053+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143053+0000","last_undegraded":"2026-02-22T20:33:33.143053+0000","last_fullsized":"2026-02-22T20:33:33.143053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143180+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143180+0000","last_peered":"2026-02-22T20:33:33.143180+0000","last_clean":"2026-02-22T20:33:33.143180+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143180+0000","last_undegraded":"2026-02-22T20:33:33.143180+0000","last_fullsized":"2026-02-22T20:33:33.143180+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705695,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705694,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:29.723 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:29.902 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:29.903 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:29.915 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2026-02-22T20:35:29.000315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001408"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:29.916 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:30.094 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:30.095 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:30.107 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2026-02-22T20:35:29.000315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001408"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:30.108 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:30.108 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:30.108 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:30.109 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:30.109 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:30.323 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:30.324 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":147,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:30.336 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:30.336 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:30.516 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:30.517 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:30.529 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2026-02-22T20:35:29.000315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001408"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143187+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143187+0000","last_peered":"2026-02-22T20:33:33.143187+0000","last_clean":"2026-02-22T20:33:33.143187+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143187+0000","last_undegraded":"2026-02-22T20:33:33.143187+0000","last_fullsized":"2026-02-22T20:33:33.143187+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705696,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705695,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:33.531 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:33.710 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:33.711 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:33.722 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2026-02-22T20:35:33.000823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:33.724 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:33.902 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:33.902 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:33.913 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2026-02-22T20:35:33.000823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:33.915 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:33.915 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:33.915 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:33.915 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:33.916 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:34.134 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:34.134 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":150,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:34.148 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:34.148 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:34.328 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:34.329 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:34.340 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2026-02-22T20:35:33.000823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705696,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:37.344 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:37.525 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:37.526 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:37.538 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2026-02-22T20:35:37.001305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001390"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:37.539 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:37.715 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:37.716 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:37.727 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2026-02-22T20:35:37.001305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001390"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:37.728 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:37.729 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:37.729 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:37.729 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:37.729 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:37.946 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:37.946 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":154,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:37.959 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:37.959 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:38.136 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:38.136 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:38.147 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2026-02-22T20:35:37.001305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001390"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705698,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705697,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:41.149 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:41.326 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:41.326 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:41.339 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2026-02-22T20:35:41.001829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001513"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:41.339 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:41.514 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:41.515 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:41.527 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2026-02-22T20:35:41.001829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001513"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:41.528 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:41.528 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:41.528 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:41.528 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:41.529 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:41.742 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:41.743 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":158,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:41.754 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:41.755 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:41.933 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:41.934 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:41.945 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2026-02-22T20:35:41.001829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001513"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:44.947 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:45.130 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:45.130 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:45.143 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2026-02-22T20:35:45.002276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001452"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:45.144 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:45.322 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:45.322 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:45.334 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2026-02-22T20:35:45.002276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001452"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:45.335 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:45.335 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:45.335 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:45.335 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:45.335 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:45.547 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:45.547 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":162,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:45.559 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:45.559 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:45.735 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:45.735 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:45.748 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2026-02-22T20:35:45.002276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001452"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705699,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705698,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:48.750 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:48.935 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:48.936 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:48.949 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2026-02-22T20:35:47.002482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001447"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:48.951 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:49.129 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:49.130 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:49.142 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2026-02-22T20:35:49.002811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001506"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:49.142 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:49.143 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:49.143 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:49.143 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:49.143 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:49.357 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:49.357 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":166,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:49.371 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:49.371 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:49.548 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:49.549 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:49.561 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2026-02-22T20:35:49.002811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001506"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705700,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705699,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:52.564 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:52.742 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:52.742 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:52.754 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2026-02-22T20:35:51.003027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001484"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:52.754 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:52.933 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:52.933 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:52.946 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2026-02-22T20:35:51.003027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001484"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:52.949 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:52.949 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:52.950 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:52.950 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:52.951 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:53.165 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:53.166 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":169,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:53.178 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:53.179 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:53.356 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:53.357 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:53.368 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2026-02-22T20:35:53.003312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001483"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:56.371 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:56.547 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:56.548 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:56.559 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2026-02-22T20:35:55.003542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001524"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:56.561 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:56.738 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:56.738 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:56.750 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2026-02-22T20:35:55.003542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001524"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705701,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705700,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:35:56.751 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:35:56.751 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:35:56.752 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:35:56.752 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:35:56.752 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:35:56.967 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:56.967 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":173,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:35:56.980 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:35:56.980 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:35:57.156 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:35:57.156 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:35:57.168 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2026-02-22T20:35:57.003869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001593"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:00.170 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:00.352 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:00.352 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:00.365 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2026-02-22T20:35:59.004091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001608"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:00.365 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:00.547 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:00.547 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:00.559 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2026-02-22T20:35:59.004091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001608"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:00.560 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:00.560 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:00.560 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:00.560 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:00.561 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:00.779 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:00.779 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":177,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:00.792 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:00.792 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:00.973 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:00.973 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:00.986 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2026-02-22T20:35:59.004091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001608"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705702,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705701,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:03.988 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:04.170 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:04.170 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:04.182 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2026-02-22T20:36:03.004540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001513"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:04.182 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:04.361 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:04.361 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:04.373 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2026-02-22T20:36:03.004540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001513"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:04.375 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:04.375 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:04.375 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:04.375 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:04.376 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:04.588 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:04.588 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":181,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:04.600 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:04.600 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:04.782 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:04.783 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:04.795 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2026-02-22T20:36:03.004540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001513"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705703,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705702,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:07.797 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:07.976 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:07.976 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:07.988 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2026-02-22T20:36:07.005031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001489"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:07.988 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:08.164 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:08.165 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:08.176 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2026-02-22T20:36:07.005031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001489"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:08.177 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:08.178 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:08.178 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:08.178 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:08.178 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:08.396 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:08.397 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":185,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:08.409 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:08.410 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:08.585 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:08.586 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:08.597 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2026-02-22T20:36:07.005031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001489"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705704,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705703,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:11.598 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:11.778 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:11.779 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:11.791 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2026-02-22T20:36:11.005500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:11.793 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:11.972 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:11.973 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:11.985 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2026-02-22T20:36:11.005500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:11.986 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:11.986 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:11.987 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:11.987 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:11.987 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:12.206 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:12.206 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":188,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:12.219 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:12.220 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:12.397 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:12.397 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:12.409 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2026-02-22T20:36:11.005500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705705,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705704,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:15.411 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:15.590 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:15.591 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:15.602 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2026-02-22T20:36:15.005987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001446"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:15.603 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:15.782 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:15.782 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:15.794 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2026-02-22T20:36:15.005987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001446"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:15.795 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:15.795 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:15.796 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:15.796 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:15.796 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:16.011 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:16.011 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":192,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:16.024 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:16.024 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:16.201 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:16.201 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:16.212 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2026-02-22T20:36:15.005987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001446"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:19.214 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:19.394 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:19.395 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:19.407 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2026-02-22T20:36:19.006430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001399"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:19.407 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:19.583 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:19.584 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:19.596 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2026-02-22T20:36:19.006430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001399"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:19.597 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:19.597 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:19.597 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:19.597 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:19.598 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:19.817 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:19.817 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":196,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:19.830 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:19.831 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:20.012 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:20.013 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:20.024 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2026-02-22T20:36:19.006430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001399"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705706,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705705,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:23.027 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:23.206 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:23.206 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:23.217 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2026-02-22T20:36:23.006915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001414"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:23.219 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:23.394 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:23.395 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:23.407 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2026-02-22T20:36:23.006915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001414"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:23.409 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:23.409 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:23.410 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:23.410 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:23.410 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:23.626 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:23.627 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":200,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:23.639 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:23.640 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:23.817 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:23.818 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:23.829 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2026-02-22T20:36:23.006915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001414"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705707,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705706,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:26.832 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:27.012 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:27.012 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:27.024 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2026-02-22T20:36:27.007280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001293"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:27.025 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:27.200 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:27.201 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:27.213 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2026-02-22T20:36:27.007280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001293"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:27.214 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:27.214 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:27.215 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:27.215 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:27.215 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:27.435 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:27.435 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":204,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:27.447 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:27.447 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:27.622 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:27.623 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:27.635 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2026-02-22T20:36:27.007280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001293"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:30.637 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:30.819 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:30.819 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:30.833 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2026-02-22T20:36:29.007509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001312"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705707,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:30.835 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:31.011 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:31.012 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:31.023 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2026-02-22T20:36:31.007780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001349"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:31.026 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:31.026 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:31.027 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:31.027 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:31.027 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:31.244 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:31.244 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":207,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:31.257 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:31.258 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:31.434 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:31.434 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:31.447 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2026-02-22T20:36:31.007780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001349"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:34.450 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:34.632 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:34.632 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:34.644 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2026-02-22T20:36:33.008013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001365"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:34.646 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:34.823 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:34.824 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:34.835 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2026-02-22T20:36:33.008013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001365"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705708,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:34.838 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:34.838 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:34.838 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:34.838 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:34.839 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:35.054 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:35.055 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":211,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:35.068 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:35.068 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:35.245 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:35.246 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:35.257 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2026-02-22T20:36:35.008277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001361"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:38.259 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:38.441 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:38.441 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:38.454 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2026-02-22T20:36:37.008491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001398"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:38.455 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:38.636 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:38.637 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:38.649 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2026-02-22T20:36:37.008491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001398"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:38.650 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:38.650 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:38.650 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:38.651 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:38.651 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:38.869 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:38.869 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":215,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:38.882 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:38.882 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:39.060 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:39.060 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:39.072 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2026-02-22T20:36:39.008752+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001472"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705710,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705709,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:42.074 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:42.251 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:42.252 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:42.263 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2026-02-22T20:36:41.008959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001450"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:42.264 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:42.441 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:42.442 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:42.454 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2026-02-22T20:36:41.008959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001450"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:42.455 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:42.455 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:42.455 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:42.456 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:42.456 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:42.667 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:42.667 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":219,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:42.680 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:42.680 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:42.857 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:42.858 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:42.870 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2026-02-22T20:36:41.008959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001450"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705711,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705710,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:45.872 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:46.055 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:46.056 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:46.068 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2026-02-22T20:36:45.009476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001462"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:46.070 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:46.248 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:46.248 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:46.260 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2026-02-22T20:36:45.009476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001462"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:46.262 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:46.262 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:46.262 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:46.262 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:46.263 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:46.479 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:46.479 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":223,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:46.492 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:46.493 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:46.669 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:46.669 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:46.681 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2026-02-22T20:36:45.009476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001462"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:49.681 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:49.862 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:49.862 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:49.874 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2026-02-22T20:36:49.009925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:49.876 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:50.053 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:50.053 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:50.065 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2026-02-22T20:36:49.009925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:50.066 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:50.066 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:50.066 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:50.066 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:50.066 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:50.280 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:50.280 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":227,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:50.293 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:50.293 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:50.470 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:50.471 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:50.483 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2026-02-22T20:36:49.009925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705712,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705711,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:53.486 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:53.664 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:53.665 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:53.676 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2026-02-22T20:36:53.010434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001474"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:53.678 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:53.858 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:53.858 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:53.869 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2026-02-22T20:36:53.010434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001474"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:53.869 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:53.870 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:53.870 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:53.870 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:53.870 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:54.086 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:54.087 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":230,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:54.099 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:54.099 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:54.277 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:54.277 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:54.289 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2026-02-22T20:36:53.010434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001474"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705713,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705712,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:57.291 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:57.472 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:57.472 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:57.485 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2026-02-22T20:36:57.010981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001505"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:57.485 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:57.664 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:57.664 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:57.676 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2026-02-22T20:36:57.010981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001505"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:36:57.676 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:36:57.676 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:36:57.676 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:36:57.676 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:36:57.677 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:36:57.891 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:57.891 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":234,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:36:57.904 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:36:57.904 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:36:58.086 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:36:58.086 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:36:58.099 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2026-02-22T20:36:57.010981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001505"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705714,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705713,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:01.100 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:01.278 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:01.278 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:01.290 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2026-02-22T20:37:01.011480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001555"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:01.291 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:01.470 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:01.470 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:01.482 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2026-02-22T20:37:01.011480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001555"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:01.483 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:01.483 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:01.483 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:01.483 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:01.483 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:01.697 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:01.697 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":238,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:01.709 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:01.709 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:01.887 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:01.888 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:01.900 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2026-02-22T20:37:01.011480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001555"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705715,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705714,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:04.902 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:05.084 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:05.084 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:05.096 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2026-02-22T20:37:05.011919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001484"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:05.096 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:05.274 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:05.275 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:05.288 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2026-02-22T20:37:05.011919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001484"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:05.289 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:05.289 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:05.290 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:05.290 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:05.290 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:05.505 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:05.505 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":242,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:05.518 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:05.518 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:05.695 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:05.696 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:05.708 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2026-02-22T20:37:05.011919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001484"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:08.710 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:08.888 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:08.888 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:08.900 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2026-02-22T20:37:07.012146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001477"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:08.900 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:09.077 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:09.077 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:09.088 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2026-02-22T20:37:09.012404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001422"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:09.089 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:09.089 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:09.089 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:09.089 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:09.089 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:09.302 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:09.302 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":246,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:09.315 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:09.315 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:09.491 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:09.491 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:09.503 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2026-02-22T20:37:09.012404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001422"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705716,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705715,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:12.504 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:12.683 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:12.683 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:12.695 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2026-02-22T20:37:11.012630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001403"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:12.696 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:12.874 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:12.874 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:12.885 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2026-02-22T20:37:11.012630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001403"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:12.886 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:12.886 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:12.886 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:12.886 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:12.886 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:13.098 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:13.098 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":249,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:13.110 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:13.111 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:13.287 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:13.288 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:13.300 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2026-02-22T20:37:13.012893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705717,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705716,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:16.301 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:16.476 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:16.476 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:16.488 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2026-02-22T20:37:15.013110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001460"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:16.489 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:16.665 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:16.665 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:16.677 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2026-02-22T20:37:15.013110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001460"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:16.678 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:16.678 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:16.678 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:16.678 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:16.678 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:16.893 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:16.893 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":253,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:16.906 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:16.906 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:17.083 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:17.084 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:17.095 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2026-02-22T20:37:17.013376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:20.097 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:20.275 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:20.275 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:20.288 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2026-02-22T20:37:19.013579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:20.289 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:20.464 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:20.465 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:20.476 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2026-02-22T20:37:19.013579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:20.477 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:20.477 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:20.478 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:20.478 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:20.478 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:20.691 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:20.691 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":257,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:20.704 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:20.704 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:20.878 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:20.878 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:20.889 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2026-02-22T20:37:19.013579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705718,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705717,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:23.890 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:24.066 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:24.066 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:24.078 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2026-02-22T20:37:23.014073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001443"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:24.078 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:24.255 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:24.256 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:24.267 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2026-02-22T20:37:23.014073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001443"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:24.269 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:24.269 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:24.269 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:24.269 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:24.270 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:24.482 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:24.482 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":261,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:24.495 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:24.496 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:24.675 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:24.675 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:24.688 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2026-02-22T20:37:23.014073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001443"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143533+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:33:33.143533+0000","last_peered":"2026-02-22T20:33:33.143533+0000","last_clean":"2026-02-22T20:33:33.143533+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:33:33.143533+0000","last_undegraded":"2026-02-22T20:33:33.143533+0000","last_fullsized":"2026-02-22T20:33:33.143533+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143245+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:33:33.143245+0000","last_peered":"2026-02-22T20:33:33.143245+0000","last_clean":"2026-02-22T20:33:33.143245+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:33:33.143245+0000","last_undegraded":"2026-02-22T20:33:33.143245+0000","last_fullsized":"2026-02-22T20:33:33.143245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143157+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:33:33.143157+0000","last_peered":"2026-02-22T20:33:33.143157+0000","last_clean":"2026-02-22T20:33:33.143157+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:33:33.143157+0000","last_undegraded":"2026-02-22T20:33:33.143157+0000","last_fullsized":"2026-02-22T20:33:33.143157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143208+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:33:33.143208+0000","last_peered":"2026-02-22T20:33:33.143208+0000","last_clean":"2026-02-22T20:33:33.143208+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:33:33.143208+0000","last_undegraded":"2026-02-22T20:33:33.143208+0000","last_fullsized":"2026-02-22T20:33:33.143208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705719,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705718,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:27.689 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:27.868 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:27.868 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:27.880 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2026-02-22T20:37:27.014582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001472"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:27.880 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:28.056 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:28.057 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:28.068 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2026-02-22T20:37:27.014582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001472"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:28.068 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:28.068 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143449+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:33:33.143449+0000', 'last_peered': '2026-02-22T20:33:33.143449+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:33:33.143449+0000', 'last_undegraded': '2026-02-22T20:33:33.143449+0000', 'last_fullsized': '2026-02-22T20:33:33.143449+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:28.068 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:28.068 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 13, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:33:33.143236+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:33:33.143236+0000', 'last_peered': '2026-02-22T20:33:33.143236+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:33:33.143236+0000', 'last_undegraded': '2026-02-22T20:33:33.143236+0000', 'last_fullsized': '2026-02-22T20:33:33.143236+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:28.068 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:28.278 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:28.279 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":265,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:28.292 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:28.292 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:28.468 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:28.468 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:28.480 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2026-02-22T20:37:27.014582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001472"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143449+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:33:33.143449+0000","last_peered":"2026-02-22T20:33:33.143449+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:33:33.143449+0000","last_undegraded":"2026-02-22T20:33:33.143449+0000","last_fullsized":"2026-02-22T20:33:33.143449+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":13,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:33:33.143236+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:33:33.143236+0000","last_peered":"2026-02-22T20:33:33.143236+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:33:33.143236+0000","last_undegraded":"2026-02-22T20:33:33.143236+0000","last_fullsized":"2026-02-22T20:33:33.143236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705720,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705719,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:31.483 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:31.663 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:31.663 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:31.676 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2026-02-22T20:37:31.015037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:31.676 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:31.853 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:31.853 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:31.865 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2026-02-22T20:37:31.015037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:31.866 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:31.866 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:31.866 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:31.866 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:31.867 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:32.081 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:32.082 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":268,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:32.094 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:32.095 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:32.271 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:32.271 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:32.284 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2026-02-22T20:37:31.015037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143328+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:33:33.143328+0000","last_peered":"2026-02-22T20:33:33.143328+0000","last_clean":"2026-02-22T20:33:33.143328+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:33:33.143328+0000","last_undegraded":"2026-02-22T20:33:33.143328+0000","last_fullsized":"2026-02-22T20:33:33.143328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705721,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705720,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:35.287 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:35.469 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:35.469 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:35.480 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2026-02-22T20:37:35.015523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001449"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:35.480 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:35.655 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:35.656 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:35.666 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2026-02-22T20:37:35.015523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001449"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:35.667 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:35.667 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:35.667 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:35.667 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:35.668 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:35.882 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:35.882 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":272,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:35.894 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:35.894 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:36.073 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:36.074 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:36.085 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2026-02-22T20:37:35.015523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001449"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:39.088 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:39.266 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:39.267 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:39.279 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2026-02-22T20:37:39.015904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:39.281 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:39.462 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:39.462 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:39.474 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2026-02-22T20:37:39.015904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:39.476 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:39.476 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:39.476 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:39.477 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:39.477 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:39.691 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:39.691 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":276,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:39.704 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:39.704 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:39.880 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:39.881 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:39.893 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2026-02-22T20:37:39.015904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:33:33.143279+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:33:33.143279+0000","last_peered":"2026-02-22T20:33:33.143279+0000","last_clean":"2026-02-22T20:33:33.143279+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:33:33.143279+0000","last_undegraded":"2026-02-22T20:33:33.143279+0000","last_fullsized":"2026-02-22T20:33:33.143279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705722,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705721,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:42.895 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:43.078 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:43.078 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:43.091 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2026-02-22T20:37:43.016385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001348"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:43.091 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:43.268 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:43.268 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:43.280 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2026-02-22T20:37:43.016385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001348"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:43.281 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:43.281 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:43.281 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:43.281 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:43.281 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:43.497 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:43.497 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":280,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:43.510 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:43.510 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:43.690 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:43.690 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:43.702 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2026-02-22T20:37:43.016385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001348"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705723,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705722,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:46.704 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:46.883 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:46.883 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:46.896 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2026-02-22T20:37:45.016581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001304"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:46.898 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:47.079 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:47.080 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:47.092 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2026-02-22T20:37:47.016804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001281"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:47.094 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:47.094 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:47.094 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:47.094 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:47.095 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:47.308 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:47.308 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":284,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:47.321 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:47.321 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:47.498 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:47.499 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:47.510 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2026-02-22T20:37:47.016804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001281"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:50.512 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:50.690 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:50.691 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:50.702 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2026-02-22T20:37:49.017031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001322"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:50.703 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:50.879 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:50.879 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:50.890 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2026-02-22T20:37:49.017031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001322"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705724,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705723,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:50.892 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:50.892 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:50.892 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:50.892 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:50.893 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:51.106 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:51.106 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":287,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:51.118 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:51.118 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:51.299 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:51.299 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:51.312 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2026-02-22T20:37:51.017277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001372"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:54.314 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:54.493 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:54.493 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:54.506 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2026-02-22T20:37:53.017498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001387"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:54.508 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:54.682 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:54.682 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:54.694 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2026-02-22T20:37:53.017498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001387"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705725,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705724,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:54.695 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:54.695 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:54.696 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:54.696 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:54.696 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:54.910 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:54.910 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":291,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:54.922 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:54.922 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:55.097 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:55.097 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:55.111 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2026-02-22T20:37:55.017771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001386"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:58.113 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:58.293 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:58.294 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:58.305 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2026-02-22T20:37:57.017983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:58.306 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:58.484 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:58.484 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:58.497 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2026-02-22T20:37:57.017983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:37:58.497 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:37:58.497 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:37:58.497 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:37:58.497 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:37:58.498 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:37:58.713 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:58.713 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":295,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:37:58.726 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:37:58.726 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:37:58.903 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:37:58.904 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:37:58.915 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2026-02-22T20:37:57.017983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705726,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705725,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:01.917 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:02.093 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:02.094 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:02.106 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2026-02-22T20:38:01.018471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001439"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:02.107 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:02.285 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:02.285 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:02.297 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2026-02-22T20:38:01.018471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001439"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:02.298 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:02.298 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:02.298 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:02.298 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:02.298 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:02.509 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:02.509 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":299,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:02.521 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:02.521 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:02.697 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:02.698 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:02.710 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2026-02-22T20:38:01.018471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001439"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705727,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705726,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:05.712 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:05.892 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:05.893 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:05.905 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2026-02-22T20:38:05.018843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001345"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:05.906 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:06.086 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:06.086 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:06.097 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2026-02-22T20:38:05.018843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001345"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:06.098 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:06.098 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:06.098 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:06.098 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:06.098 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:06.319 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:06.319 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":303,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:06.332 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:06.332 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:06.505 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:06.505 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:06.517 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2026-02-22T20:38:05.018843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001345"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:09.518 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:09.698 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:09.698 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:09.710 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2026-02-22T20:38:09.019331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001348"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:09.710 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:09.885 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:09.885 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:09.897 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2026-02-22T20:38:09.019331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001348"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:09.897 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:09.897 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:09.897 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:09.897 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:09.897 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:10.115 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:10.115 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":306,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:10.130 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:10.130 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:10.306 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:10.306 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:10.319 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2026-02-22T20:38:09.019331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001348"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705728,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705727,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:13.321 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:13.499 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:13.499 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:13.511 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2026-02-22T20:38:13.019802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001331"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:13.511 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:13.691 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:13.691 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:13.703 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2026-02-22T20:38:13.019802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001331"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:13.704 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:13.704 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:13.705 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:13.705 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:13.705 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:13.922 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:13.922 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":310,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:13.934 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:13.935 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:14.113 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:14.114 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:14.127 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2026-02-22T20:38:13.019802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001331"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705729,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705728,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:17.129 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:17.309 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:17.310 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:17.322 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2026-02-22T20:38:17.020287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001444"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:17.324 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:17.500 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:17.500 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:17.511 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2026-02-22T20:38:17.020287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001444"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:17.511 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:17.511 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:17.511 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:17.511 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:17.512 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:17.724 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:17.725 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":314,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:17.738 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:17.739 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:17.918 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:17.919 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:17.930 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2026-02-22T20:38:17.020287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001444"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705730,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705729,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:20.932 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:21.114 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:21.114 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:21.126 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2026-02-22T20:38:21.020733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:21.128 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:21.306 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:21.307 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:21.318 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2026-02-22T20:38:21.020733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:21.319 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:21.320 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:21.320 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:21.320 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:21.320 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:21.533 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:21.534 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":318,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:21.546 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:21.546 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:21.723 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:21.723 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:21.735 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2026-02-22T20:38:21.020733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:24.736 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:24.912 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:24.913 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:24.924 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2026-02-22T20:38:23.020934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001397"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705731,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705730,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:24.926 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:25.103 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:25.103 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:25.115 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2026-02-22T20:38:25.021169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:25.117 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:25.117 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:25.117 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:25.117 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:25.118 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:25.334 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:25.334 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":322,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:25.347 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:25.348 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:25.525 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:25.525 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:25.537 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2026-02-22T20:38:25.021169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:28.540 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:28.716 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:28.717 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:28.729 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2026-02-22T20:38:27.021380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001363"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:28.729 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:28.907 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:28.907 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:28.919 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2026-02-22T20:38:27.021380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001363"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:28.920 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:28.920 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:28.920 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:28.920 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:28.920 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:29.135 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:29.135 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":325,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:29.149 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:29.149 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:29.325 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:29.325 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:29.336 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2026-02-22T20:38:29.021624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001336"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705732,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705731,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:32.337 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:32.515 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:32.515 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:32.527 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2026-02-22T20:38:31.021823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001314"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:32.528 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:32.705 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:32.705 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:32.717 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2026-02-22T20:38:31.021823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001314"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:32.718 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:32.718 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:32.719 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:32.719 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:32.719 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:32.932 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:32.932 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":329,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:32.945 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:32.945 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:33.123 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:33.123 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:33.136 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2026-02-22T20:38:33.022068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001335"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705733,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705732,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:36.137 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:36.313 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:36.313 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:36.325 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2026-02-22T20:38:35.022274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001340"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:36.326 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:36.504 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:36.505 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:36.516 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2026-02-22T20:38:35.022274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001340"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:36.518 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:36.518 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:36.518 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:36.518 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:36.519 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:36.734 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:36.734 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":333,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:36.747 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:36.747 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:36.925 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:36.925 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:36.938 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2026-02-22T20:38:35.022274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001340"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:39.941 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:40.120 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:40.121 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:40.133 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2026-02-22T20:38:39.022656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001276"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:40.134 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:40.311 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:40.312 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:40.323 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2026-02-22T20:38:39.022656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001276"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:40.323 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:40.323 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:40.323 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:40.324 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:40.324 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:40.543 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:40.543 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":337,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:40.556 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:40.556 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:40.733 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:40.733 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:40.746 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2026-02-22T20:38:39.022656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001276"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705734,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705733,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:43.747 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:43.931 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:43.932 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:43.943 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2026-02-22T20:38:43.023162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001339"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:43.944 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:44.124 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:44.125 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:44.138 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2026-02-22T20:38:43.023162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001339"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:44.139 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:44.140 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:44.140 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:44.140 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:44.140 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:44.353 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:44.354 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":341,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:44.365 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:44.365 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:44.541 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:44.542 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:44.553 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2026-02-22T20:38:43.023162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001339"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705735,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705734,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:47.554 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:47.734 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:47.735 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:47.747 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2026-02-22T20:38:47.023563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001289"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:47.747 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:47.925 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:47.926 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:47.939 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2026-02-22T20:38:47.023563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001289"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:47.939 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:47.939 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:47.939 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:47.939 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:47.939 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:48.154 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:48.154 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":344,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:48.168 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:48.168 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:48.346 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:48.346 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:48.358 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2026-02-22T20:38:47.023563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001289"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705736,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705735,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:51.360 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:51.544 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:51.544 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:51.556 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2026-02-22T20:38:51.024078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:51.557 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:51.736 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:51.736 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:51.748 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2026-02-22T20:38:51.024078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:51.748 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:51.748 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:51.749 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:51.749 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:51.749 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:51.962 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:51.962 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":348,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:51.974 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:51.975 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:52.153 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:52.153 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:52.165 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2026-02-22T20:38:51.024078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705737,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705736,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:55.167 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:55.344 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:55.344 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:55.356 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2026-02-22T20:38:55.024492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001330"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:55.358 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:55.535 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:55.536 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:55.548 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2026-02-22T20:38:55.024492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001330"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:55.551 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:55.551 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:55.552 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:55.552 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:55.552 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:55.765 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:55.766 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":352,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:55.778 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:55.778 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:55.956 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:55.956 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:55.968 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2026-02-22T20:38:55.024492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001330"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:58.970 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:59.149 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:59.149 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:59.162 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2026-02-22T20:38:59.024956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:59.162 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:59.338 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:59.338 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:59.350 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2026-02-22T20:38:59.024956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:38:59.350 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:38:59.351 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:38:59.351 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:38:59.351 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:38:59.351 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:38:59.569 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:59.569 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":356,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:38:59.582 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:38:59.582 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:38:59.758 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:38:59.759 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:38:59.770 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2026-02-22T20:38:59.024956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705738,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705737,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:02.771 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:02.949 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:02.950 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:02.960 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2026-02-22T20:39:01.025170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001383"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:02.962 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:03.140 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:03.140 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:03.152 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2026-02-22T20:39:03.025387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001309"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:03.152 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:03.153 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:03.153 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:03.153 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:03.153 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:03.367 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:03.367 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":360,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:03.380 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:03.380 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:03.559 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:03.560 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:03.571 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2026-02-22T20:39:03.025387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001309"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705739,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705738,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:06.573 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:06.752 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:06.753 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:06.764 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2026-02-22T20:39:05.025629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001320"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:06.765 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:06.946 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:06.947 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:06.958 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2026-02-22T20:39:05.025629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001320"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:06.960 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:06.960 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:06.960 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:06.960 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:06.961 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:07.173 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:07.174 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":363,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:07.188 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:07.188 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:07.366 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:07.366 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:07.378 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2026-02-22T20:39:07.025910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001417"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:10.379 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:10.558 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:10.559 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:10.570 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2026-02-22T20:39:09.026126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001410"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:10.572 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:10.748 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:10.748 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:10.760 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2026-02-22T20:39:09.026126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001410"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705740,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705739,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:10.761 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:10.761 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:10.761 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:10.761 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:10.761 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:10.979 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:10.979 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":367,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:10.993 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:10.993 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:11.172 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:11.172 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:11.184 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2026-02-22T20:39:11.026377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:14.186 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:14.364 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:14.365 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:14.376 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2026-02-22T20:39:13.026603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:14.377 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:14.553 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:14.553 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:14.565 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2026-02-22T20:39:13.026603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:14.566 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:14.566 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:14.566 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:14.566 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:14.567 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:14.781 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:14.781 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":371,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:14.794 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:14.794 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:14.973 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:14.973 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:14.986 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2026-02-22T20:39:13.026603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705741,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705740,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:17.987 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:18.169 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:18.170 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:18.182 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2026-02-22T20:39:17.027061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:18.184 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:18.359 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:18.360 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:18.371 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2026-02-22T20:39:17.027061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:18.372 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:18.372 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:18.372 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:18.372 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:18.372 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:18.587 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:18.587 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":375,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:18.599 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:18.600 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:18.775 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:18.775 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:18.787 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2026-02-22T20:39:17.027061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705742,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705741,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:21.789 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:21.968 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:21.968 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:21.980 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2026-02-22T20:39:21.027534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001407"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:21.981 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:22.160 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:22.160 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:22.172 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2026-02-22T20:39:21.027534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001407"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:22.174 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:22.174 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:22.174 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:22.174 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:22.175 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:22.392 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:22.392 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":379,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:22.405 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:22.405 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:22.584 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:22.584 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:22.596 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2026-02-22T20:39:21.027534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001407"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001410+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:35:25.001410+0000","last_peered":"2026-02-22T20:35:25.001410+0000","last_clean":"2026-02-22T20:35:25.001410+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:35:25.001410+0000","last_undegraded":"2026-02-22T20:35:25.001410+0000","last_fullsized":"2026-02-22T20:35:25.001410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705743,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705742,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:25.598 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:25.779 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:25.779 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:25.792 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2026-02-22T20:39:25.028009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001405"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:25.793 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:25.971 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:25.972 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:25.983 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2026-02-22T20:39:25.028009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001405"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:25.985 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:25.985 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:25.985 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:25.985 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:25.986 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:26.197 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:26.197 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":382,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:26.210 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:26.211 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:26.387 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:26.387 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:26.399 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2026-02-22T20:39:25.028009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001405"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:29.401 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:29.586 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:29.587 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:29.598 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2026-02-22T20:39:29.028495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:29.599 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:29.779 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:29.779 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:29.792 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2026-02-22T20:39:29.028495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:29.792 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:29.792 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:29.792 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:29.792 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:29.792 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:30.010 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:30.010 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":386,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:30.022 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:30.023 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:30.202 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:30.202 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:30.215 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2026-02-22T20:39:29.028495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000970+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:35:20.000970+0000","last_peered":"2026-02-22T20:35:20.000970+0000","last_clean":"2026-02-22T20:35:20.000970+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:35:20.000970+0000","last_undegraded":"2026-02-22T20:35:20.000970+0000","last_fullsized":"2026-02-22T20:35:20.000970+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000613+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:35:20.000613+0000","last_peered":"2026-02-22T20:35:20.000613+0000","last_clean":"2026-02-22T20:35:20.000613+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:35:20.000613+0000","last_undegraded":"2026-02-22T20:35:20.000613+0000","last_fullsized":"2026-02-22T20:35:20.000613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:20.000580+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:35:20.000580+0000","last_peered":"2026-02-22T20:35:20.000580+0000","last_clean":"2026-02-22T20:35:20.000580+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:35:20.000580+0000","last_undegraded":"2026-02-22T20:35:20.000580+0000","last_fullsized":"2026-02-22T20:35:20.000580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705744,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705743,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:33.217 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:33.397 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:33.397 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:33.409 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2026-02-22T20:39:33.028961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001427"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:33.410 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:33.588 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:33.589 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:33.600 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2026-02-22T20:39:33.028961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001427"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:33.601 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:33.602 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 21, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001437+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:35:25.001437+0000', 'last_peered': '2026-02-22T20:35:25.001437+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:35:25.001437+0000', 'last_undegraded': '2026-02-22T20:35:25.001437+0000', 'last_fullsized': '2026-02-22T20:35:25.001437+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:33.602 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:33.602 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 14, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:35:25.001080+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:35:25.001080+0000', 'last_peered': '2026-02-22T20:35:25.001080+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:35:25.001080+0000', 'last_undegraded': '2026-02-22T20:35:25.001080+0000', 'last_fullsized': '2026-02-22T20:35:25.001080+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:33.602 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:33.816 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:33.817 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":390,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:33.828 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:33.829 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:34.004 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:34.004 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:34.016 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2026-02-22T20:39:33.028961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001427"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":21,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001437+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:35:25.001437+0000","last_peered":"2026-02-22T20:35:25.001437+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:35:25.001437+0000","last_undegraded":"2026-02-22T20:35:25.001437+0000","last_fullsized":"2026-02-22T20:35:25.001437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":14,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:35:25.001080+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:35:25.001080+0000","last_peered":"2026-02-22T20:35:25.001080+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:35:25.001080+0000","last_undegraded":"2026-02-22T20:35:25.001080+0000","last_fullsized":"2026-02-22T20:35:25.001080+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705745,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705744,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:37.017 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:37.194 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:37.195 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:37.206 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2026-02-22T20:39:37.029431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001422"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:37.207 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:37.382 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:37.382 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:37.393 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2026-02-22T20:39:37.029431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001422"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:37.394 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:37.394 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:37.394 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:37.394 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:37.394 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:37.607 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:37.607 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":394,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:37.620 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:37.620 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:37.797 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:37.797 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:37.810 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2026-02-22T20:39:37.029431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001422"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:40.811 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:40.990 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:40.991 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:41.002 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2026-02-22T20:39:39.029616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001373"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705746,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705745,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:41.002 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:41.179 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:41.179 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:41.191 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2026-02-22T20:39:41.029873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001377"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:41.193 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:41.193 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:41.193 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:41.193 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:41.194 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:41.408 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:41.408 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":398,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:41.420 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:41.420 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:41.596 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:41.596 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:41.608 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2026-02-22T20:39:41.029873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001377"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:44.610 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:44.793 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:44.794 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:44.807 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2026-02-22T20:39:43.030062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001352"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:44.808 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:44.986 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:44.986 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:44.998 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2026-02-22T20:39:43.030062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001352"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:25.001156+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:35:25.001156+0000","last_peered":"2026-02-22T20:35:25.001156+0000","last_clean":"2026-02-22T20:35:25.001156+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:35:25.001156+0000","last_undegraded":"2026-02-22T20:35:25.001156+0000","last_fullsized":"2026-02-22T20:35:25.001156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705747,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705746,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:44.999 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:45.000 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:45.000 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:45.000 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:45.000 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:45.215 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:45.215 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":401,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:45.228 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:45.229 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:45.405 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:45.406 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:45.417 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2026-02-22T20:39:45.030318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001357"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:48.419 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:48.616 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:48.616 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:48.629 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2026-02-22T20:39:47.030546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001364"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:48.631 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:48.809 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:48.810 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:48.822 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2026-02-22T20:39:47.030546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001364"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:48.823 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:48.823 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:48.824 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:48.824 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:48.824 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:49.042 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:49.042 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":405,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:49.055 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:49.055 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:49.232 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:49.232 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:49.247 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2026-02-22T20:39:49.030806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001374"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:35:30.001426+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:35:30.001426+0000","last_peered":"2026-02-22T20:35:30.001426+0000","last_clean":"2026-02-22T20:35:30.001426+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:35:30.001426+0000","last_undegraded":"2026-02-22T20:35:30.001426+0000","last_fullsized":"2026-02-22T20:35:30.001426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705748,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705747,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:52.248 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:52.432 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:52.432 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:52.444 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2026-02-22T20:39:51.031005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001389"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:52.445 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:52.622 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:52.623 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:52.635 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2026-02-22T20:39:51.031005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001389"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:52.636 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:52.636 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:52.637 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:52.637 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:52.637 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:52.855 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:52.855 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":409,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:52.867 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:52.867 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:53.045 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:53.046 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:53.058 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2026-02-22T20:39:53.031236+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001363"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705749,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705748,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:56.059 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:56.241 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:56.241 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:56.254 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2026-02-22T20:39:55.031445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001383"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:56.254 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:56.432 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:56.433 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:56.445 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2026-02-22T20:39:55.031445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001383"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:56.446 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:39:56.446 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:39:56.446 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:39:56.446 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:39:56.446 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:39:56.665 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:56.666 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":413,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:39:56.679 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:39:56.679 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:39:56.854 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:39:56.855 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:39:56.866 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2026-02-22T20:39:55.031445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001383"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:39:59.868 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:00.047 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:00.047 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:00.059 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2026-02-22T20:39:59.031897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001350"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:00.060 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:00.242 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:00.242 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:00.254 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2026-02-22T20:39:59.031897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001350"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:00.256 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:00.256 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:00.256 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:00.256 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:00.256 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:00.471 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:00.472 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":417,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:00.484 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:00.485 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:00.663 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:00.663 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:00.675 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2026-02-22T20:39:59.031897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001350"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705750,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:03.676 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:03.854 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:03.855 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:03.864 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2026-02-22T20:40:03.032319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001314"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:03.865 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:04.047 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:04.047 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:04.059 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2026-02-22T20:40:03.032319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001314"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:04.060 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:04.060 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:04.060 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:04.060 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:04.060 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:04.281 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:04.281 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":421,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:04.293 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:04.293 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:04.467 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:04.467 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:04.478 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2026-02-22T20:40:03.032319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001314"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705750,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:07.480 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:07.657 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:07.658 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:07.669 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2026-02-22T20:40:07.032770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001325"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:07.671 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:07.850 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:07.850 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:07.861 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2026-02-22T20:40:07.032770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001325"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:07.862 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:07.862 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:07.862 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:07.862 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:07.862 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:08.077 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:08.077 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":424,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:08.091 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:08.091 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:08.269 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:08.269 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:08.282 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2026-02-22T20:40:07.032770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001325"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705752,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705751,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:11.283 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:11.457 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:11.457 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:11.468 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2026-02-22T20:40:11.033152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001255"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:11.469 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:11.646 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:11.646 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:11.658 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2026-02-22T20:40:11.033152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001255"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:11.660 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:11.660 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:11.660 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:11.660 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:11.661 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:11.877 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:11.877 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":428,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:11.889 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:11.890 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:12.069 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:12.069 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:12.081 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2026-02-22T20:40:11.033152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001255"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705753,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705752,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:15.082 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:15.264 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:15.264 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:15.277 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2026-02-22T20:40:15.033614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001295"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:15.278 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:15.453 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:15.454 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:15.465 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2026-02-22T20:40:15.033614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001295"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:15.466 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:15.467 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:15.467 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:15.467 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:15.468 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:15.683 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:15.683 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":432,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:15.695 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:15.695 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:15.872 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:15.873 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:15.885 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2026-02-22T20:40:15.033614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001295"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:18.887 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:19.066 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:19.066 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:19.078 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2026-02-22T20:40:19.034074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001303"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:19.088 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:19.264 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:19.264 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:19.277 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2026-02-22T20:40:19.034074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001303"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:19.278 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:19.278 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:19.278 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:19.278 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:19.278 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:19.493 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:19.493 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":436,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:19.507 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:19.507 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:19.686 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:19.687 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:19.699 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2026-02-22T20:40:19.034074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001303"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705754,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705753,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:22.701 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:22.878 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:22.878 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:22.890 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2026-02-22T20:40:21.034266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001284"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:22.890 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:23.066 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:23.066 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:23.079 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2026-02-22T20:40:23.034478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001325"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:23.079 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:23.080 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:23.080 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:23.080 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:23.080 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:23.292 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:23.292 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":440,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:23.305 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:23.305 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:23.485 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:23.485 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:23.496 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2026-02-22T20:40:23.034478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001325"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705755,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705754,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:26.498 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:26.678 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:26.679 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:26.691 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2026-02-22T20:40:25.034691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001342"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:26.691 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:26.866 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:26.866 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:26.878 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2026-02-22T20:40:25.034691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001342"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:26.879 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:26.879 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:26.879 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:26.879 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:26.880 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:27.096 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:27.096 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":443,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:27.110 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:27.110 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:27.286 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:27.286 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:27.298 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2026-02-22T20:40:27.034903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001288"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:30.299 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:30.477 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:30.477 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:30.489 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2026-02-22T20:40:29.035116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001278"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:30.489 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:30.667 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:30.668 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:30.679 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2026-02-22T20:40:29.035116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001278"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705756,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705755,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:30.680 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:30.680 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:30.680 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:30.681 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:30.681 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:30.896 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:30.896 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":447,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:30.908 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:30.909 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:31.089 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:31.090 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:31.102 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2026-02-22T20:40:31.035356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001281"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:34.104 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:34.284 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:34.284 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:34.296 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2026-02-22T20:40:33.035562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001295"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:34.297 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:34.475 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:34.476 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:34.487 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2026-02-22T20:40:33.035562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001295"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:34.488 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:34.488 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:34.488 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:34.488 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:34.488 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:34.702 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:34.702 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":451,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:34.714 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:34.715 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:34.894 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:34.894 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:34.905 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2026-02-22T20:40:33.035562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001295"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705757,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705756,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:37.907 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:38.087 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:38.087 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:38.099 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2026-02-22T20:40:37.035996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001305"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:38.100 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:38.280 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:38.281 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:38.293 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2026-02-22T20:40:37.035996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001305"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:38.294 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:38.294 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:38.294 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:38.294 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:38.295 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:38.511 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:38.511 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":455,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:38.524 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:38.524 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:38.708 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:38.709 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:38.721 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2026-02-22T20:40:37.035996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001305"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705758,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:41.723 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:41.904 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:41.905 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:41.916 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2026-02-22T20:40:41.036480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001363"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:41.917 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:42.097 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:42.098 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:42.110 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2026-02-22T20:40:41.036480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001363"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:42.111 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:42.111 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:42.111 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:42.112 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:42.112 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:42.329 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:42.329 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":459,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:42.341 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:42.341 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:42.518 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:42.518 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:42.530 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2026-02-22T20:40:41.036480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001363"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705759,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705758,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:45.532 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:45.710 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:45.710 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:45.722 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2026-02-22T20:40:45.037002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001440"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:45.722 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:45.900 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:45.901 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:45.912 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2026-02-22T20:40:45.037002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001440"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:45.913 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:45.913 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:45.913 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:45.913 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:45.914 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:46.126 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:46.126 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":462,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:46.138 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:46.138 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:46.319 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:46.320 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:46.332 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2026-02-22T20:40:45.037002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001440"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:49.334 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:49.515 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:49.516 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:49.528 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2026-02-22T20:40:49.037492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001496"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:49.530 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:49.707 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:49.708 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:49.720 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2026-02-22T20:40:49.037492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001496"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:49.720 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:49.720 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:49.720 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:49.720 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:49.720 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:49.934 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:49.934 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":466,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:49.947 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:49.948 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:50.126 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:50.126 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:50.138 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2026-02-22T20:40:49.037492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001496"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705760,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705759,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:53.139 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:53.315 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:53.315 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:53.328 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2026-02-22T20:40:53.037987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001507"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:53.328 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:53.504 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:53.504 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:53.516 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2026-02-22T20:40:53.037987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001507"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:53.518 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:53.518 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:53.518 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:53.518 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:53.519 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:53.734 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:53.734 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":470,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:53.746 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:53.746 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:53.922 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:53.922 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:53.935 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2026-02-22T20:40:53.037987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001507"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705761,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705760,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:56.936 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:57.114 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:57.114 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:57.126 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2026-02-22T20:40:57.038518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001516"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:57.127 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:57.304 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:57.305 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:57.317 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2026-02-22T20:40:57.038518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001516"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:40:57.317 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:40:57.317 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:40:57.317 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:40:57.317 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:40:57.318 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:40:57.531 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:57.531 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":474,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:40:57.543 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:40:57.544 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:40:57.721 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:40:57.722 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:40:57.735 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2026-02-22T20:40:57.038518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001516"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:00.737 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:00.916 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:00.917 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:00.930 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2026-02-22T20:40:59.038705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001458"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705762,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705761,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:00.931 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:01.110 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:01.111 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:01.122 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2026-02-22T20:41:01.038942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001449"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:01.124 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:01.124 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:01.124 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:01.124 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:01.124 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:01.338 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:01.339 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":478,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:01.351 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:01.351 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:01.529 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:01.530 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:01.542 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2026-02-22T20:41:01.038942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001449"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:04.543 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:04.720 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:04.720 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:04.733 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2026-02-22T20:41:03.039163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001437"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:04.733 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:04.910 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:04.911 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:04.922 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2026-02-22T20:41:03.039163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001437"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705763,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705762,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:04.923 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:04.923 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:04.924 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:04.924 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:04.924 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:05.139 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:05.139 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":481,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:05.151 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:05.152 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:05.326 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:05.326 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:05.337 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2026-02-22T20:41:05.039414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001426"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:08.338 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:08.516 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:08.516 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:08.527 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2026-02-22T20:41:07.039643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:08.528 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:08.705 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:08.706 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:08.717 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2026-02-22T20:41:07.039643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001409"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:08.719 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:08.719 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:08.719 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:08.719 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:08.719 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:08.933 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:08.933 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":485,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:08.946 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:08.946 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:09.121 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:09.122 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:09.134 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2026-02-22T20:41:09.039893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001374"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705764,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705763,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:12.136 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:12.316 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:12.317 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:12.329 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2026-02-22T20:41:11.040111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:12.330 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:12.508 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:12.509 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:12.521 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2026-02-22T20:41:11.040111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:12.523 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:12.523 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:12.523 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:12.523 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:12.524 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:12.744 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:12.744 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":489,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:12.756 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:12.756 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:12.935 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:12.935 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:12.948 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2026-02-22T20:41:11.040111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705765,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705764,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:15.949 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:16.129 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:16.130 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:16.141 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2026-02-22T20:41:15.040574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001411"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:16.143 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:16.321 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:16.322 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:16.334 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2026-02-22T20:41:15.040574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001411"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:16.334 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:16.334 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:16.335 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:16.335 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:16.335 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:16.553 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:16.553 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":493,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:16.567 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:16.568 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:16.746 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:16.747 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:16.758 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2026-02-22T20:41:15.040574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001411"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:19.759 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:19.939 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:19.940 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:19.952 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2026-02-22T20:41:19.041004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001360"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:19.952 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:20.128 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:20.129 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:20.140 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2026-02-22T20:41:19.041004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001360"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:20.141 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:20.141 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:20.141 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:20.141 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:20.141 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:20.353 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:20.354 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":497,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:20.367 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:20.367 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:20.545 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:20.545 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:20.557 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2026-02-22T20:41:19.041004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001360"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705766,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705765,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:23.558 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:23.739 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:23.740 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:23.752 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2026-02-22T20:41:23.041475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001364"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:23.754 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:23.930 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:23.930 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:23.943 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2026-02-22T20:41:23.041475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001364"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:23.943 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:23.943 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:23.943 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:23.943 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:23.944 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:24.156 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:24.157 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":500,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:24.169 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:24.169 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:24.348 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:24.348 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:24.360 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2026-02-22T20:41:23.041475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001364"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011258+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:37:25.011258+0000","last_peered":"2026-02-22T20:37:25.011258+0000","last_clean":"2026-02-22T20:37:25.011258+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:37:25.011258+0000","last_undegraded":"2026-02-22T20:37:25.011258+0000","last_fullsized":"2026-02-22T20:37:25.011258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705767,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705766,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:27.361 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:27.541 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:27.541 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:27.553 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2026-02-22T20:41:27.041977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:27.553 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:27.729 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:27.730 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:27.742 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2026-02-22T20:41:27.041977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:27.743 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:27.743 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:27.743 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:27.743 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:27.744 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:27.960 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:27.961 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":504,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:27.974 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:27.974 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:28.151 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:28.151 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:28.162 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2026-02-22T20:41:27.041977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705768,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:31.165 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:31.343 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:31.344 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:31.355 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2026-02-22T20:41:31.042450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001445"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:31.356 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:31.533 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:31.533 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:31.545 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2026-02-22T20:41:31.042450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001445"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:31.545 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:31.545 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:31.545 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:31.545 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:31.546 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:31.759 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:31.760 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":508,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:31.771 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:31.772 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:31.948 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:31.949 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:31.960 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2026-02-22T20:41:31.042450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001445"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011279+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:37:25.011279+0000","last_peered":"2026-02-22T20:37:25.011279+0000","last_clean":"2026-02-22T20:37:25.011279+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:37:25.011279+0000","last_undegraded":"2026-02-22T20:37:25.011279+0000","last_fullsized":"2026-02-22T20:37:25.011279+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011166+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:37:25.011166+0000","last_peered":"2026-02-22T20:37:25.011166+0000","last_clean":"2026-02-22T20:37:25.011166+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:37:25.011166+0000","last_undegraded":"2026-02-22T20:37:25.011166+0000","last_fullsized":"2026-02-22T20:37:25.011166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:25.011135+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:37:25.011135+0000","last_peered":"2026-02-22T20:37:25.011135+0000","last_clean":"2026-02-22T20:37:25.011135+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:37:25.011135+0000","last_undegraded":"2026-02-22T20:37:25.011135+0000","last_fullsized":"2026-02-22T20:37:25.011135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705769,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705768,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:34.962 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:35.143 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:35.144 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:35.156 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2026-02-22T20:41:35.042908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:35.159 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:35.336 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:35.337 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:35.349 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2026-02-22T20:41:35.042908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:35.350 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:35.350 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:35.351 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:35.351 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:35.351 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:35.564 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:35.565 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":512,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:35.576 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:35.576 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:35.750 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:35.751 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:35.762 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2026-02-22T20:41:35.042908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:38.764 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:38.944 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:38.945 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:38.956 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2026-02-22T20:41:37.043122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001404"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:38.958 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:39.134 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:39.135 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:39.147 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2026-02-22T20:41:39.043380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:39.147 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:39.147 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 22, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011558+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:37:30.011558+0000', 'last_peered': '2026-02-22T20:37:30.011558+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:37:30.011558+0000', 'last_undegraded': '2026-02-22T20:37:30.011558+0000', 'last_fullsized': '2026-02-22T20:37:30.011558+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:39.148 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:39.148 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 15, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:37:30.011450+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:37:30.011450+0000', 'last_peered': '2026-02-22T20:37:30.011450+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:37:30.011450+0000', 'last_undegraded': '2026-02-22T20:37:30.011450+0000', 'last_fullsized': '2026-02-22T20:37:30.011450+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:39.148 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:39.360 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:39.360 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":516,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:39.372 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:39.373 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:39.550 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:39.551 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:39.562 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2026-02-22T20:41:39.043380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":22,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011558+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:37:30.011558+0000","last_peered":"2026-02-22T20:37:30.011558+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:37:30.011558+0000","last_undegraded":"2026-02-22T20:37:30.011558+0000","last_fullsized":"2026-02-22T20:37:30.011558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":15,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:37:30.011450+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:37:30.011450+0000","last_peered":"2026-02-22T20:37:30.011450+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:37:30.011450+0000","last_undegraded":"2026-02-22T20:37:30.011450+0000","last_fullsized":"2026-02-22T20:37:30.011450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705770,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:42.563 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:42.740 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:42.740 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:42.752 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2026-02-22T20:41:41.043590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001381"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:42.752 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:42.930 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:42.931 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:42.943 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2026-02-22T20:41:41.043590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001381"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:42.945 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:42.945 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:42.945 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:42.945 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:42.946 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:43.161 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:43.161 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":519,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:43.173 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:43.173 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:43.352 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:43.352 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:43.364 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2026-02-22T20:41:43.043861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001411"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705771,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705770,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:46.366 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:46.541 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:46.541 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:46.552 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2026-02-22T20:41:45.044038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001375"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:46.553 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:46.728 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:46.728 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:46.740 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2026-02-22T20:41:45.044038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001375"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:46.741 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:46.741 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:46.741 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:46.741 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:46.741 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:46.953 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:46.953 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":523,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:46.967 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:46.968 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:47.146 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:47.146 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:47.158 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2026-02-22T20:41:47.044245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001337"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:50.159 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:50.338 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:50.338 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:50.350 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2026-02-22T20:41:49.044455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001333"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:50.352 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:50.531 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:50.532 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:50.544 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2026-02-22T20:41:49.044455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001333"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:50.545 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:50.546 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:50.546 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:50.546 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:50.546 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:50.760 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:50.761 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":527,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:50.772 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:50.772 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:50.949 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:50.950 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:50.961 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2026-02-22T20:41:49.044455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001333"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705772,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705771,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:53.963 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:54.142 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:54.142 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:54.156 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2026-02-22T20:41:53.044948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001358"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:54.157 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:54.336 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:54.336 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:54.348 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2026-02-22T20:41:53.044948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001358"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:54.349 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:54.349 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:54.350 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:54.350 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:54.350 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:54.564 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:54.564 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":531,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:54.576 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:54.576 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:54.749 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:54.749 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:54.760 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2026-02-22T20:41:53.044948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001358"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:35.011722+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:37:35.011722+0000","last_peered":"2026-02-22T20:37:35.011722+0000","last_clean":"2026-02-22T20:37:35.011722+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:37:35.011722+0000","last_undegraded":"2026-02-22T20:37:35.011722+0000","last_fullsized":"2026-02-22T20:37:35.011722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705773,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705772,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:57.762 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:57.944 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:57.944 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:57.956 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2026-02-22T20:41:57.045454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:57.957 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:58.136 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:58.136 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:58.148 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2026-02-22T20:41:57.045454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:41:58.149 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:41:58.149 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:41:58.149 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:41:58.149 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:41:58.149 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:41:58.362 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:58.363 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":535,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:41:58.375 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:41:58.375 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:41:58.552 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:41:58.552 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:41:58.564 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2026-02-22T20:41:57.045454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:37:40.011979+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:37:40.011979+0000","last_peered":"2026-02-22T20:37:40.011979+0000","last_clean":"2026-02-22T20:37:40.011979+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:37:40.011979+0000","last_undegraded":"2026-02-22T20:37:40.011979+0000","last_fullsized":"2026-02-22T20:37:40.011979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705774,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:01.565 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:01.744 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:01.744 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:01.756 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2026-02-22T20:42:01.045911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001455"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:01.756 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:01.930 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:01.931 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:01.943 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2026-02-22T20:42:01.045911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001455"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:01.944 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:01.944 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:01.944 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:01.945 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:01.945 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:02.159 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:02.159 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":538,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:02.172 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:02.172 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:02.349 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:02.349 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:02.361 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2026-02-22T20:42:01.045911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001455"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705775,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705774,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:05.363 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:05.544 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:05.544 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:05.556 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2026-02-22T20:42:05.046335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001387"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:05.557 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:05.732 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:05.732 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:05.744 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2026-02-22T20:42:05.046335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001387"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:05.746 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:05.746 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:05.747 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:05.747 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:05.747 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:05.961 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:05.961 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":542,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:05.974 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:05.974 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:06.152 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:06.153 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:06.166 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2026-02-22T20:42:05.046335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001387"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:09.167 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:09.352 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:09.353 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:09.365 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2026-02-22T20:42:09.046721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001267"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:09.366 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:09.544 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:09.545 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:09.557 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2026-02-22T20:42:09.046721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001267"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:09.558 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:09.558 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:09.558 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:09.558 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:09.558 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:09.775 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:09.776 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":546,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:09.788 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:09.789 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:09.967 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:09.967 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:09.978 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2026-02-22T20:42:09.046721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001267"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705776,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:12.979 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:13.157 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:13.158 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:13.170 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2026-02-22T20:42:13.047087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001175"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:13.171 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:13.350 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:13.350 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:13.364 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2026-02-22T20:42:13.047087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001175"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:13.367 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:13.367 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:13.367 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:13.368 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:13.368 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:13.587 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:13.587 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":550,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:13.601 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:13.601 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:13.779 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:13.779 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:13.791 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2026-02-22T20:42:13.047087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001175"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705777,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705776,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:16.792 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:16.970 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:16.971 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:16.982 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2026-02-22T20:42:15.047320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001168"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:16.983 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:17.157 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:17.157 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:17.170 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2026-02-22T20:42:17.047573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001237"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:17.171 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:17.171 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:17.171 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:17.171 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:17.172 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:17.384 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:17.384 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":554,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:17.396 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:17.397 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:17.574 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:17.575 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:17.587 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2026-02-22T20:42:17.047573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001237"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:20.589 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:20.767 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:20.767 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:20.779 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2026-02-22T20:42:19.047808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001291"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:20.780 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:20.957 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:20.957 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:20.969 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2026-02-22T20:42:19.047808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001291"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705778,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705777,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:20.970 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:20.970 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:20.970 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:20.970 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:20.970 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:21.185 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:21.185 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":557,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:21.198 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:21.199 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:21.382 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:21.382 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:21.394 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2026-02-22T20:42:21.048049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001327"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:24.396 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:24.576 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:24.576 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:24.589 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2026-02-22T20:42:23.048252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001364"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:24.589 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:24.764 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:24.764 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:24.777 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2026-02-22T20:42:23.048252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001364"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705779,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:24.778 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:24.778 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:24.779 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:24.779 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:24.779 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:24.994 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:24.994 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":561,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:25.007 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:25.007 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:25.190 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:25.191 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:25.203 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2026-02-22T20:42:25.048502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:28.204 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:28.383 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:28.383 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:28.394 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2026-02-22T20:42:27.048744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001423"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:28.395 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:28.571 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:28.571 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:28.584 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2026-02-22T20:42:27.048744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001423"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:28.584 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:28.584 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:28.584 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:28.584 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:28.584 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:28.801 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:28.801 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":565,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:28.815 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:28.816 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:28.994 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:28.995 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:29.006 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2026-02-22T20:42:27.048744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001423"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705780,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705779,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:32.008 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:32.185 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:32.186 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:32.198 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2026-02-22T20:42:31.049233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001424"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:32.198 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:32.374 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:32.374 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:32.385 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2026-02-22T20:42:31.049233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001424"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:32.386 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:32.386 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:32.386 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:32.386 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:32.386 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:32.601 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:32.601 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":569,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:32.613 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:32.613 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:32.790 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:32.791 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:32.803 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2026-02-22T20:42:31.049233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001424"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705781,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:35.805 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:35.985 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:35.986 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:35.998 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2026-02-22T20:42:35.049739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001487"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:35.999 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:36.176 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:36.177 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:36.189 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2026-02-22T20:42:35.049739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001487"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:36.190 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:36.190 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:36.190 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:36.190 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:36.190 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:36.405 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:36.405 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":573,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:36.418 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:36.418 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:36.593 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:36.593 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:36.606 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2026-02-22T20:42:35.049739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001487"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:39.607 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:39.787 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:39.787 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:39.799 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2026-02-22T20:42:39.050170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001425"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:39.799 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:39.981 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:39.981 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:39.993 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2026-02-22T20:42:39.050170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001425"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:39.994 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:39.995 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:39.995 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:39.995 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:39.995 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:40.208 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:40.208 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":576,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:40.221 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:40.221 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:40.399 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:40.399 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:40.411 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2026-02-22T20:42:39.050170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001425"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705782,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705781,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:43.413 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:43.591 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:43.591 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:43.603 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2026-02-22T20:42:43.050663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001430"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:43.604 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:43.786 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:43.786 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:43.798 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2026-02-22T20:42:43.050663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001430"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:43.798 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:43.798 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:43.798 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:43.799 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:43.799 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:44.015 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:44.015 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":580,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:44.026 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:44.027 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:44.205 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:44.205 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:44.217 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2026-02-22T20:42:43.050663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001430"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705783,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705782,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:47.219 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:47.399 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:47.399 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:47.410 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2026-02-22T20:42:47.051159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001419"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:47.411 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:47.586 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:47.586 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:47.597 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2026-02-22T20:42:47.051159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001419"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:47.598 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:47.598 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:47.598 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:47.598 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:47.598 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:47.814 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:47.814 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":584,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:47.827 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:47.827 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:48.003 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:48.003 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:48.015 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2026-02-22T20:42:47.051159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001419"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705784,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705783,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:51.016 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:51.195 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:51.196 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:51.208 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2026-02-22T20:42:51.051685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001514"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:51.211 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:51.389 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:51.390 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:51.402 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2026-02-22T20:42:51.051685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001514"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:51.403 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:51.403 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:51.404 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:51.404 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:51.404 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:51.623 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:51.623 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":588,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:51.635 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:51.635 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:51.817 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:51.818 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:51.830 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2026-02-22T20:42:51.051685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001514"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:54.832 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:55.012 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:55.013 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:55.025 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2026-02-22T20:42:53.051912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001525"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705785,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705784,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:55.026 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:55.204 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:55.205 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:55.217 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2026-02-22T20:42:55.052176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001512"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:55.218 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:55.218 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:55.218 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:55.218 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:55.219 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:55.433 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:55.434 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":592,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:55.446 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:55.446 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:55.638 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:55.639 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:55.651 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2026-02-22T20:42:55.052176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001512"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:58.654 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:58.834 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:58.834 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:58.847 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2026-02-22T20:42:57.052384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:58.849 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:59.026 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:59.026 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:59.038 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2026-02-22T20:42:57.052384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:42:59.040 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:42:59.040 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:42:59.040 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:42:59.040 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:42:59.040 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:42:59.254 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:59.254 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":595,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:42:59.267 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:42:59.267 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:42:59.442 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:42:59.442 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:42:59.453 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2026-02-22T20:42:59.052636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001477"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705786,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:02.456 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:02.634 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:02.634 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:02.647 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2026-02-22T20:43:01.052834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001425"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:02.648 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:02.824 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:02.824 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:02.837 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2026-02-22T20:43:01.052834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001425"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:02.837 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:02.837 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:02.837 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:02.837 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:02.838 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:03.050 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:03.051 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":599,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:03.062 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:03.062 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:03.238 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:03.238 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:03.250 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2026-02-22T20:43:03.053093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001408"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705787,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705786,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:06.251 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:06.430 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:06.430 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:06.442 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2026-02-22T20:43:05.053290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001378"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:06.443 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:06.620 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:06.620 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:06.632 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2026-02-22T20:43:05.053290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001378"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:06.633 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:06.633 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:06.633 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:06.633 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:06.633 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:06.844 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:06.845 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":603,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:06.856 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:06.857 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:07.033 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:07.034 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:07.045 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2026-02-22T20:43:05.053290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001378"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:10.047 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:10.223 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:10.223 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:10.236 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2026-02-22T20:43:09.053722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001338"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:10.236 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:10.415 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:10.416 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:10.428 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2026-02-22T20:43:09.053722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001338"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:10.429 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:10.429 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:10.430 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:10.430 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:10.430 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:10.646 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:10.646 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":607,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:10.659 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:10.659 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:10.837 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:10.837 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:10.850 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2026-02-22T20:43:09.053722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001338"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705788,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705787,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:13.851 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:14.029 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:14.030 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:14.042 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2026-02-22T20:43:13.054228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001394"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705789,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:14.044 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:14.223 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:14.223 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:14.237 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2026-02-22T20:43:13.054228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001394"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705789,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:14.237 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:14.237 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:14.237 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:14.237 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:14.237 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:14.452 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:14.453 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":611,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:14.465 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:14.466 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:14.644 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:14.644 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:14.656 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2026-02-22T20:43:13.054228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001394"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705789,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705789,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:17.657 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:17.841 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:17.841 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:17.853 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2026-02-22T20:43:17.054685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001394"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705789,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:17.854 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:18.032 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:18.032 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:18.045 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2026-02-22T20:43:17.054685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001394"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705789,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:18.046 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:18.046 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:18.046 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:18.046 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:18.046 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:18.259 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:18.260 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":614,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:18.272 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:18.272 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:18.450 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:18.450 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:18.461 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2026-02-22T20:43:17.054685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001394"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705790,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705789,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:21.463 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:21.640 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:21.640 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:21.652 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2026-02-22T20:43:21.055180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:21.652 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:21.826 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:21.827 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:21.838 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2026-02-22T20:43:21.055180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:21.840 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:21.840 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:21.840 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:21.840 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:21.840 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:22.053 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:22.053 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":618,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:22.066 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:22.066 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:22.241 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:22.241 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:22.253 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2026-02-22T20:43:21.055180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:25.019877+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:39:25.019877+0000","last_peered":"2026-02-22T20:39:25.019877+0000","last_clean":"2026-02-22T20:39:25.019877+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:39:25.019877+0000","last_undegraded":"2026-02-22T20:39:25.019877+0000","last_fullsized":"2026-02-22T20:39:25.019877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705791,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705790,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:25.254 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:25.434 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:25.434 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:25.446 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2026-02-22T20:43:25.055686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001458"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:25.447 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:25.623 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:25.623 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:25.635 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2026-02-22T20:43:25.055686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001458"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:25.635 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:25.635 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:25.636 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:25.636 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:25.636 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:25.850 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:25.851 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":622,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:25.864 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:25.864 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:26.040 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:26.040 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:26.052 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2026-02-22T20:43:25.055686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001458"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:29.053 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:29.233 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:29.233 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:29.247 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2026-02-22T20:43:29.056181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001495"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:29.248 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:29.428 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:29.429 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:29.440 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2026-02-22T20:43:29.056181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001495"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:29.442 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:29.442 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:29.442 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:29.442 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:29.443 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:29.660 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:29.661 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":626,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:29.674 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:29.674 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:29.855 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:29.855 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:29.866 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2026-02-22T20:43:29.056181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001495"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705792,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705791,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:32.867 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:33.047 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:33.048 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:33.060 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2026-02-22T20:43:31.056391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001459"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:33.062 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:33.241 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:33.241 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:33.254 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2026-02-22T20:43:33.056592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:33.254 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:33.254 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:33.254 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:33.254 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:33.254 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:33.468 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:33.469 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":630,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:33.481 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:33.481 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:33.660 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:33.660 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:33.673 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2026-02-22T20:43:33.056592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705793,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705792,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:36.675 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:36.853 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:36.854 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:36.866 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2026-02-22T20:43:35.056801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001382"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:36.866 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:37.044 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:37.044 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:37.057 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2026-02-22T20:43:35.056801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001382"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:37.058 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:37.058 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:37.058 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:37.058 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:37.059 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:37.274 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:37.275 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":634,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:37.290 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:37.291 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:37.473 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:37.474 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:37.487 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2026-02-22T20:43:37.057081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001394"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:40.490 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:40.669 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:40.669 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:40.682 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2026-02-22T20:43:39.057257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001349"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:40.682 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:40.863 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:40.863 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:40.875 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2026-02-22T20:43:39.057257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001349"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020145+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:39:30.020145+0000","last_peered":"2026-02-22T20:39:30.020145+0000","last_clean":"2026-02-22T20:39:30.020145+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:39:30.020145+0000","last_undegraded":"2026-02-22T20:39:30.020145+0000","last_fullsized":"2026-02-22T20:39:30.020145+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020031+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:39:30.020031+0000","last_peered":"2026-02-22T20:39:30.020031+0000","last_clean":"2026-02-22T20:39:30.020031+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:39:30.020031+0000","last_undegraded":"2026-02-22T20:39:30.020031+0000","last_fullsized":"2026-02-22T20:39:30.020031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:30.020014+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:39:30.020014+0000","last_peered":"2026-02-22T20:39:30.020014+0000","last_clean":"2026-02-22T20:39:30.020014+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:39:30.020014+0000","last_undegraded":"2026-02-22T20:39:30.020014+0000","last_fullsized":"2026-02-22T20:39:30.020014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705794,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:40.876 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:40.876 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:40.876 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:40.876 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:40.876 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:41.094 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:41.094 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":637,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:41.107 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:41.107 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:41.285 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:41.286 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:41.297 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2026-02-22T20:43:41.057503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001322"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:44.298 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:44.484 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:44.484 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:44.497 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2026-02-22T20:43:43.057716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001324"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:44.499 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:44.678 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:44.678 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:44.690 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2026-02-22T20:43:43.057716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001324"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":23,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020631+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:39:35.020631+0000","last_peered":"2026-02-22T20:39:35.020631+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:39:35.020631+0000","last_undegraded":"2026-02-22T20:39:35.020631+0000","last_fullsized":"2026-02-22T20:39:35.020631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":16,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:39:35.020479+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:39:35.020479+0000","last_peered":"2026-02-22T20:39:35.020479+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:39:35.020479+0000","last_undegraded":"2026-02-22T20:39:35.020479+0000","last_fullsized":"2026-02-22T20:39:35.020479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705795,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705794,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:44.691 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:44.691 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 23, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020631+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:39:35.020631+0000', 'last_peered': '2026-02-22T20:39:35.020631+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:39:35.020631+0000', 'last_undegraded': '2026-02-22T20:39:35.020631+0000', 'last_fullsized': '2026-02-22T20:39:35.020631+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:44.691 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:44.691 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 16, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:39:35.020479+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:39:35.020479+0000', 'last_peered': '2026-02-22T20:39:35.020479+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:39:35.020479+0000', 'last_undegraded': '2026-02-22T20:39:35.020479+0000', 'last_fullsized': '2026-02-22T20:39:35.020479+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:44.691 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:44.901 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:44.902 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":641,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:44.914 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:44.914 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:45.093 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:45.094 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:45.106 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2026-02-22T20:43:45.057967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001374"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:48.107 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:48.287 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:48.288 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:48.299 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2026-02-22T20:43:47.058178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001376"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:48.299 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:48.478 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:48.479 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:48.492 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2026-02-22T20:43:47.058178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001376"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:48.493 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:48.493 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:48.493 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:48.494 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:48.494 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:48.706 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:48.707 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":645,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:48.718 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:48.718 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:48.893 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:48.893 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:48.905 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2026-02-22T20:43:47.058178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001376"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705796,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:51.906 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:52.085 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:52.086 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:52.097 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2026-02-22T20:43:51.058564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001306"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:52.099 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:52.277 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:52.277 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:52.289 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2026-02-22T20:43:51.058564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001306"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:52.289 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:52.289 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:52.289 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:52.290 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:52.290 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:52.510 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:52.510 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":649,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:52.524 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:52.524 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:52.703 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:52.704 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:52.716 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2026-02-22T20:43:51.058564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001306"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705797,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705796,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:55.719 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:55.901 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:55.901 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:55.914 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2026-02-22T20:43:55.058996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001280"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:55.916 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:56.093 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:56.094 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:56.106 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2026-02-22T20:43:55.058996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001280"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:56.107 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:56.107 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:56.107 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:56.107 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:56.107 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:43:56.320 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:56.320 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":653,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:43:56.332 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:43:56.332 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:56.511 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:56.512 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:56.523 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2026-02-22T20:43:55.058996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001280"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:59.526 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:59.708 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:59.708 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:59.721 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2026-02-22T20:43:59.059478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001299"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:59.723 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:43:59.898 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:43:59.898 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:43:59.910 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2026-02-22T20:43:59.059478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001299"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:43:59.910 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:43:59.910 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:43:59.910 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:43:59.911 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:43:59.911 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:00.127 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:00.127 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":656,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:00.139 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:00.140 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:00.322 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:00.323 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:00.334 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2026-02-22T20:43:59.059478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001299"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705798,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705797,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:03.334 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:03.513 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:03.514 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:03.525 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2026-02-22T20:44:03.059985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:03.525 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:03.702 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:03.702 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:03.713 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2026-02-22T20:44:03.059985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:03.714 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:03.714 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:03.714 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:03.714 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:03.714 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:03.927 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:03.927 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":660,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:03.940 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:03.940 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:04.117 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:04.117 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:04.129 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2026-02-22T20:44:03.059985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:45.021248+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:39:45.021248+0000","last_peered":"2026-02-22T20:39:45.021248+0000","last_clean":"2026-02-22T20:39:45.021248+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:39:45.021248+0000","last_undegraded":"2026-02-22T20:39:45.021248+0000","last_fullsized":"2026-02-22T20:39:45.021248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705799,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:07.131 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:07.310 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:07.310 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:07.323 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2026-02-22T20:44:07.060340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001344"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:07.324 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:07.505 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:07.506 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:07.518 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2026-02-22T20:44:07.060340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001344"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:07.519 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:07.519 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:07.520 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:07.520 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:07.520 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:07.734 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:07.735 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":664,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:07.747 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:07.747 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:07.923 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:07.923 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:07.936 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2026-02-22T20:44:07.060340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001344"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:39:50.021772+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:39:50.021772+0000","last_peered":"2026-02-22T20:39:50.021772+0000","last_clean":"2026-02-22T20:39:50.021772+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:39:50.021772+0000","last_undegraded":"2026-02-22T20:39:50.021772+0000","last_fullsized":"2026-02-22T20:39:50.021772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705800,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705799,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:10.937 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:11.116 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:11.117 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:11.129 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2026-02-22T20:44:11.060781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001303"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:11.130 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:11.308 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:11.309 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:11.321 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2026-02-22T20:44:11.060781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001303"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:11.321 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:11.322 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:11.322 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:11.322 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:11.322 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:11.535 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:11.537 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":668,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:11.548 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:11.548 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:11.725 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:11.725 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:11.736 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2026-02-22T20:44:11.060781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001303"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:14.738 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:14.916 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:14.916 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:14.928 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2026-02-22T20:44:13.061024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001289"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705801,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705800,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:14.928 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:15.102 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:15.103 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:15.115 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2026-02-22T20:44:15.061258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001272"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:15.116 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:15.116 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:15.116 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:15.116 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:15.117 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:15.330 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:15.330 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":672,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:15.343 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:15.343 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:15.521 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:15.521 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:15.533 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2026-02-22T20:44:15.061258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001272"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:18.534 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:18.715 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:18.715 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:18.727 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2026-02-22T20:44:17.061451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001314"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:18.728 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:18.905 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:18.906 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:18.917 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2026-02-22T20:44:17.061451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001314"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:18.918 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:18.918 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:18.918 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:18.918 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:18.918 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:19.133 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:19.133 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":675,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:19.146 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:19.146 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:19.323 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:19.323 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:19.335 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2026-02-22T20:44:19.061633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001292"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705802,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705801,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:22.336 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:22.518 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:22.518 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:22.531 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2026-02-22T20:44:21.061815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001289"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:22.532 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:22.710 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:22.711 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:22.722 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2026-02-22T20:44:21.061815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001289"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:22.723 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:22.723 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:22.723 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:22.723 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:22.723 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:22.937 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:22.937 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":679,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:22.950 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:22.950 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:23.126 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:23.127 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:23.138 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2026-02-22T20:44:23.062023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001241"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705803,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705802,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:26.140 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:26.319 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:26.320 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:26.331 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2026-02-22T20:44:25.062213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001188"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:26.333 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:26.512 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:26.512 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:26.523 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2026-02-22T20:44:25.062213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001188"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:26.524 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:26.524 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:26.524 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:26.524 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:26.524 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:26.742 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:26.742 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":683,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:26.755 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:26.755 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:26.932 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:26.933 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:26.945 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2026-02-22T20:44:25.062213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001188"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:29.946 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:30.125 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:30.125 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:30.136 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2026-02-22T20:44:29.062566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001114"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:30.137 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:30.313 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:30.314 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:30.326 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2026-02-22T20:44:29.062566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001114"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:30.326 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:30.326 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:30.327 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:30.327 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:30.327 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:30.541 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:30.541 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":687,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:30.554 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:30.554 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:30.732 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:30.732 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:30.744 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2026-02-22T20:44:29.062566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001114"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705804,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705803,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:33.746 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:33.926 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:33.926 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:33.939 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2026-02-22T20:44:33.062990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001175"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:33.939 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:34.121 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:34.121 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:34.133 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2026-02-22T20:44:33.062990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001175"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:34.133 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:34.133 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:34.133 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:34.133 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:34.134 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:34.350 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:34.351 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":691,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:34.363 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:34.364 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:34.543 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:34.543 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:34.554 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2026-02-22T20:44:33.062990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001175"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705805,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705804,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:37.555 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:37.732 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:37.732 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:37.743 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2026-02-22T20:44:37.063475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001261"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:37.744 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:37.919 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:37.919 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:37.931 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2026-02-22T20:44:37.063475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001261"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:37.931 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:37.932 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:37.932 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:37.932 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:37.932 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:38.143 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:38.143 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":694,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:38.155 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:38.156 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:38.333 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:38.333 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:38.346 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2026-02-22T20:44:37.063475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001261"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705806,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705805,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:41.347 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:41.528 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:41.528 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:41.540 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2026-02-22T20:44:41.064048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001482"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:41.542 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:41.721 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:41.721 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:41.734 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2026-02-22T20:44:41.064048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001482"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:41.735 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:41.735 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:41.736 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:41.736 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:41.736 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:41.950 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:41.950 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":698,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:41.963 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:41.964 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:42.141 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:42.141 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:42.152 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2026-02-22T20:44:41.064048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001482"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705807,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705806,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:45.153 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:45.335 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:45.335 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:45.348 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2026-02-22T20:44:45.064518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001528"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:45.349 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:45.525 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:45.525 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:45.537 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2026-02-22T20:44:45.064518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001528"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:45.538 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:45.538 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:45.538 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:45.539 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:45.539 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:45.753 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:45.753 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":702,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:45.766 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:45.766 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:45.941 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:45.942 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:45.953 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2026-02-22T20:44:45.064518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001528"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:48.957 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:49.135 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:49.136 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:49.148 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2026-02-22T20:44:49.064986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001511"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:49.150 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:49.328 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:49.328 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:49.340 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2026-02-22T20:44:49.064986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001511"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:49.341 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:49.341 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:49.341 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:49.341 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:49.341 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:49.554 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:49.554 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":706,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:49.566 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:49.566 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:49.745 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:49.745 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:49.757 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2026-02-22T20:44:49.064986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001511"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705808,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705807,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:52.759 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:52.939 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:52.939 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:52.951 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2026-02-22T20:44:51.065196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001416"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:52.952 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:53.129 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:53.129 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:53.141 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2026-02-22T20:44:53.065455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:53.141 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:53.142 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:53.142 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:53.142 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:53.142 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:53.366 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:53.366 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":710,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:53.379 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:53.379 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:53.555 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:53.555 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:53.568 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2026-02-22T20:44:53.065455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001406"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705809,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705808,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:56.569 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:56.745 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:56.746 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:56.757 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2026-02-22T20:44:55.065667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001385"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:56.758 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:56.932 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:56.932 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:56.944 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2026-02-22T20:44:55.065667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001385"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:44:56.945 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:44:56.945 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:44:56.945 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:44:56.945 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:44:56.945 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:44:57.160 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:57.160 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":713,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:44:57.173 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:44:57.173 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:44:57.348 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:44:57.349 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:44:57.360 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2026-02-22T20:44:57.065948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:00.362 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:00.544 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:00.544 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:00.557 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2026-02-22T20:44:59.066153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:00.558 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:00.737 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:00.737 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:00.750 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2026-02-22T20:44:59.066153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705810,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705809,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:00.752 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:00.752 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:00.752 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:00.752 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:00.753 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:00.972 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:00.973 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":717,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:00.986 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:00.986 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:01.162 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:01.162 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:01.176 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2026-02-22T20:45:01.066378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:04.180 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:04.358 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:04.358 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:04.370 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2026-02-22T20:45:03.066613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001417"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:04.370 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:04.549 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:04.549 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:04.561 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2026-02-22T20:45:03.066613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001417"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:04.562 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:04.562 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:04.562 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:04.562 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:04.563 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:04.779 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:04.779 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":721,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:04.792 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:04.792 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:04.969 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:04.970 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:04.982 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2026-02-22T20:45:03.066613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001417"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705811,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705810,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:07.984 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:08.162 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:08.163 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:08.175 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2026-02-22T20:45:07.067048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001380"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:08.176 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:08.353 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:08.353 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:08.365 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2026-02-22T20:45:07.067048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001380"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:08.365 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:08.365 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:08.365 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:08.365 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:08.365 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:08.577 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:08.577 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":725,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:08.589 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:08.589 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:08.767 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:08.767 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:08.779 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2026-02-22T20:45:07.067048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001380"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705812,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:11.781 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:11.958 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:11.959 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:11.971 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2026-02-22T20:45:11.067522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001368"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:11.972 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:12.149 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:12.150 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:12.163 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2026-02-22T20:45:11.067522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001368"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:12.165 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:12.165 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:12.165 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:12.165 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:12.166 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:12.380 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:12.381 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":729,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:12.393 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:12.394 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:12.572 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:12.573 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:12.585 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2026-02-22T20:45:11.067522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001368"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705813,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705812,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:15.587 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:15.764 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:15.765 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:15.776 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2026-02-22T20:45:15.067947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001333"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:15.777 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:15.952 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:15.952 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:15.964 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2026-02-22T20:45:15.067947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001333"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:15.965 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:15.966 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:15.966 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:15.966 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:15.966 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:16.179 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:16.179 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":732,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:16.192 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:16.192 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:16.371 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:16.371 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:16.382 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2026-02-22T20:45:15.067947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001333"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:19.384 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:19.561 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:19.561 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:19.574 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2026-02-22T20:45:19.068414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001366"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:19.576 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:19.752 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:19.752 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:19.765 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2026-02-22T20:45:19.068414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001366"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:19.766 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:19.766 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:19.766 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:19.766 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:19.766 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:19.978 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:19.979 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":736,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:19.991 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:19.991 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:20.169 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:20.170 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:20.184 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2026-02-22T20:45:19.068414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001366"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705814,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:23.186 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:23.368 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:23.368 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:23.380 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2026-02-22T20:45:23.068959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001437"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:23.381 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:23.558 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:23.559 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:23.570 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2026-02-22T20:45:23.068959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001437"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:23.572 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:23.572 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:23.572 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:23.573 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:23.573 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:23.784 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:23.784 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":740,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:23.797 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:23.797 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:23.977 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:23.978 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:23.989 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2026-02-22T20:45:23.068959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001437"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":18,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:25.029152+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:41:25.029152+0000","last_peered":"2026-02-22T20:41:25.029152+0000","last_clean":"2026-02-22T20:41:25.029152+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:41:25.029152+0000","last_undegraded":"2026-02-22T20:41:25.029152+0000","last_fullsized":"2026-02-22T20:41:25.029152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705815,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705814,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:26.992 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:27.173 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:27.173 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:27.187 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2026-02-22T20:45:27.069416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:27.188 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:27.363 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:27.364 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:27.375 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2026-02-22T20:45:27.069416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:27.377 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:27.377 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:27.377 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:27.377 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:27.378 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:27.594 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:27.594 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":744,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:27.607 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:27.607 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:27.783 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:27.783 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:27.795 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2026-02-22T20:45:27.069416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:30.796 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:30.978 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:30.979 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:30.990 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2026-02-22T20:45:29.069641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001489"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705816,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:30.992 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:31.167 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:31.168 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:31.180 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2026-02-22T20:45:31.069884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001470"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:31.181 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:31.181 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:31.182 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:31.182 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:31.182 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:31.397 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:31.397 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":748,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:31.409 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:31.410 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:31.586 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:31.586 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:31.598 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2026-02-22T20:45:31.069884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001470"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:34.599 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:34.778 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:34.779 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:34.790 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2026-02-22T20:45:33.070089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:34.791 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:34.969 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:34.970 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:34.981 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2026-02-22T20:45:33.070089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705816,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:34.983 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:34.983 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:34.983 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:34.983 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:34.984 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:35.198 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:35.199 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":751,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:35.211 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:35.211 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:35.388 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:35.389 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:35.402 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2026-02-22T20:45:35.070346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001386"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:38.403 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:38.580 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:38.581 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:38.592 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2026-02-22T20:45:37.070538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001336"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:38.594 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:38.772 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:38.772 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:38.784 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2026-02-22T20:45:37.070538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001336"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:38.786 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:38.786 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:38.786 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:38.786 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:38.787 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:39.000 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:39.000 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":755,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:39.012 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:39.012 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:39.187 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:39.188 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:39.201 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2026-02-22T20:45:39.070741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001324"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705818,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:42.203 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:42.379 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:42.380 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:42.392 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2026-02-22T20:45:41.070929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001288"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:42.393 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:42.570 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:42.571 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:42.582 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2026-02-22T20:45:41.070929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001288"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:42.583 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:42.583 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:42.583 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:42.583 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:42.584 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:42.795 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:42.795 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":759,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:42.807 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:42.807 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:42.988 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:42.989 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:43.000 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2026-02-22T20:45:41.070929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001288"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029903+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:41:35.029903+0000","last_peered":"2026-02-22T20:41:35.029903+0000","last_clean":"2026-02-22T20:41:35.029903+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:41:35.029903+0000","last_undegraded":"2026-02-22T20:41:35.029903+0000","last_fullsized":"2026-02-22T20:41:35.029903+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029994+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:41:35.029994+0000","last_peered":"2026-02-22T20:41:35.029994+0000","last_clean":"2026-02-22T20:41:35.029994+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:41:35.029994+0000","last_undegraded":"2026-02-22T20:41:35.029994+0000","last_fullsized":"2026-02-22T20:41:35.029994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:35.029982+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:41:35.029982+0000","last_peered":"2026-02-22T20:41:35.029982+0000","last_clean":"2026-02-22T20:41:35.029982+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:41:35.029982+0000","last_undegraded":"2026-02-22T20:41:35.029982+0000","last_fullsized":"2026-02-22T20:41:35.029982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705819,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705818,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:46.001 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:46.178 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:46.178 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:46.191 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2026-02-22T20:45:45.071282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001192"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:46.192 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:46.372 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:46.372 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:46.385 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2026-02-22T20:45:45.071282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001192"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:46.387 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:46.387 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:46.388 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:46.388 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:46.389 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:46.611 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:46.612 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":763,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:46.625 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:46.625 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:46.803 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:46.804 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:46.815 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2026-02-22T20:45:45.071282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001192"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:49.817 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:49.998 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:49.998 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:50.010 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2026-02-22T20:45:49.071660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001122"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:50.011 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:50.189 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:50.190 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:50.203 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2026-02-22T20:45:49.071660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001122"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:50.205 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:50.205 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 24, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030141+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:41:40.030141+0000', 'last_peered': '2026-02-22T20:41:40.030141+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:41:40.030141+0000', 'last_undegraded': '2026-02-22T20:41:40.030141+0000', 'last_fullsized': '2026-02-22T20:41:40.030141+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:50.206 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:50.206 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 17, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:41:40.030274+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:41:40.030274+0000', 'last_peered': '2026-02-22T20:41:40.030274+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:41:40.030274+0000', 'last_undegraded': '2026-02-22T20:41:40.030274+0000', 'last_fullsized': '2026-02-22T20:41:40.030274+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:50.206 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:50.421 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:50.421 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":767,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:50.433 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:50.434 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:50.611 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:50.611 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:50.623 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2026-02-22T20:45:49.071660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001122"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":24,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030141+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:41:40.030141+0000","last_peered":"2026-02-22T20:41:40.030141+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:41:40.030141+0000","last_undegraded":"2026-02-22T20:41:40.030141+0000","last_fullsized":"2026-02-22T20:41:40.030141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":17,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:41:40.030274+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:41:40.030274+0000","last_peered":"2026-02-22T20:41:40.030274+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:41:40.030274+0000","last_undegraded":"2026-02-22T20:41:40.030274+0000","last_fullsized":"2026-02-22T20:41:40.030274+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705819,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:53.625 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:53.808 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:53.808 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:53.822 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2026-02-22T20:45:53.072116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001186"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:53.823 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:54.000 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:54.000 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:54.011 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2026-02-22T20:45:53.072116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001186"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:54.011 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:54.011 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:54.012 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:54.012 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:54.012 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:54.228 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:54.228 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":770,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:54.241 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:54.241 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:54.419 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:54.419 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:54.431 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2026-02-22T20:45:53.072116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001186"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705820,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:57.433 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:57.614 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:57.615 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:57.627 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2026-02-22T20:45:57.072557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001275"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:57.628 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:57.806 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:57.806 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:57.818 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2026-02-22T20:45:57.072557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001275"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:45:57.818 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:45:57.818 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:45:57.819 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:45:57.819 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:45:57.819 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:45:58.031 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:58.032 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":774,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:45:58.044 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:45:58.044 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:45:58.220 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:45:58.221 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:45:58.233 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2026-02-22T20:45:57.072557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001275"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:01.235 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:01.415 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:01.415 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:01.427 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2026-02-22T20:46:01.072934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001274"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:01.428 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:01.603 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:01.603 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:01.614 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2026-02-22T20:46:01.072934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001274"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:01.615 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:01.615 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:01.615 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:01.615 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:01.615 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:01.831 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:01.832 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":778,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:01.844 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:01.844 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:02.021 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:02.021 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:02.032 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2026-02-22T20:46:01.072934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001274"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705823,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:05.034 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:05.212 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:05.212 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:05.225 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2026-02-22T20:46:05.073413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001296"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:05.225 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:05.402 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:05.403 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:05.414 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2026-02-22T20:46:05.073413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001296"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:05.416 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:05.416 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:05.417 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:05.417 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:05.418 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:05.631 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:05.631 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":782,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:05.643 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:05.644 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:05.819 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:05.820 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:05.830 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2026-02-22T20:46:05.073413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001296"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:08.832 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:09.010 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:09.010 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:09.021 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2026-02-22T20:46:07.073622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001269"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:09.022 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:09.201 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:09.202 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:09.214 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2026-02-22T20:46:09.073872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001314"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:09.215 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:09.216 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:09.216 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:09.216 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:09.216 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:09.430 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:09.430 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":786,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:09.442 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:09.443 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:09.619 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:09.620 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:09.632 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2026-02-22T20:46:09.073872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001314"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:12.634 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:12.811 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:12.811 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:12.822 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2026-02-22T20:46:11.074096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001355"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:12.824 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:13.002 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:13.002 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:13.014 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2026-02-22T20:46:11.074096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001355"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:13.015 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:13.015 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:13.015 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:13.015 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:13.015 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:13.230 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:13.230 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":789,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:13.244 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:13.244 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:13.421 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:13.421 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:13.433 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2026-02-22T20:46:13.074379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001444"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":39,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:41:55.031471+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:41:55.031471+0000","last_peered":"2026-02-22T20:41:55.031471+0000","last_clean":"2026-02-22T20:41:55.031471+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:41:55.031471+0000","last_undegraded":"2026-02-22T20:41:55.031471+0000","last_fullsized":"2026-02-22T20:41:55.031471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:16.434 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:16.612 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:16.612 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:16.625 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2026-02-22T20:46:15.074579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001414"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:16.627 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:16.803 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:16.803 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:16.815 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2026-02-22T20:46:15.074579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001414"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:16.816 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:16.816 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:16.816 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:16.816 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:16.816 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:17.028 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:17.028 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":793,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:17.041 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:17.041 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:17.218 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:17.218 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:17.231 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2026-02-22T20:46:17.074810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001397"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:20.233 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:20.411 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:20.412 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:20.424 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2026-02-22T20:46:19.075021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001399"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:20.426 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:20.603 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:20.604 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:20.617 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2026-02-22T20:46:19.075021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001399"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:20.618 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:20.619 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:20.619 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:20.619 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:20.620 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:20.835 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:20.835 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":797,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:20.848 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:20.848 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:21.025 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:21.025 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:21.036 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2026-02-22T20:46:19.075021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001399"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:42:00.031915+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:42:00.031915+0000","last_peered":"2026-02-22T20:42:00.031915+0000","last_clean":"2026-02-22T20:42:00.031915+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:42:00.031915+0000","last_undegraded":"2026-02-22T20:42:00.031915+0000","last_fullsized":"2026-02-22T20:42:00.031915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705826,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:24.039 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:24.217 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:24.217 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:24.228 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2026-02-22T20:46:23.075518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001422"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:24.230 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:24.408 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:24.408 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:24.420 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2026-02-22T20:46:23.075518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001422"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:24.421 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:24.422 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:24.422 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:24.422 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:24.423 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:24.639 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:24.639 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":801,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:24.652 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:24.653 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:24.832 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:24.832 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:24.844 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2026-02-22T20:46:23.075518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001422"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705827,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705826,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:27.845 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:28.024 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:28.025 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:28.037 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2026-02-22T20:46:27.076010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001431"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:28.038 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:28.217 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:28.217 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:28.228 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2026-02-22T20:46:27.076010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001431"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:28.229 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:28.229 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:28.229 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:28.229 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:28.230 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:28.442 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:28.442 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":805,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:28.455 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:28.455 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:28.633 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:28.634 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:28.645 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2026-02-22T20:46:27.076010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001431"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705827,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:31.647 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:31.828 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:31.829 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:31.841 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2026-02-22T20:46:31.076478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001456"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:31.842 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:32.020 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:32.020 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:32.031 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2026-02-22T20:46:31.076478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001456"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:32.032 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:32.032 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:32.032 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:32.032 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:32.032 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:32.246 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:32.246 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":808,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:32.260 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:32.261 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:32.439 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:32.439 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:32.451 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2026-02-22T20:46:31.076478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001456"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705829,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:35.454 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:35.633 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:35.634 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:35.646 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2026-02-22T20:46:35.077014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001495"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:35.647 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:35.824 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:35.825 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:35.836 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2026-02-22T20:46:35.077014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001495"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:35.838 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:35.838 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:35.838 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:35.838 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:35.839 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:36.055 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:36.055 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":812,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:36.068 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:36.069 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:36.246 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:36.247 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:36.258 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2026-02-22T20:46:35.077014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001495"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:39.261 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:39.447 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:39.447 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:39.461 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2026-02-22T20:46:39.077439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:39.463 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:39.639 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:39.639 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:39.651 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2026-02-22T20:46:39.077439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:39.652 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:39.652 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:39.652 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:39.652 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:39.652 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:39.869 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:39.869 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":816,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:39.882 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:39.882 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:40.060 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:40.061 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:40.073 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2026-02-22T20:46:39.077439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705830,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705829,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:43.075 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:43.257 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:43.258 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:43.270 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2026-02-22T20:46:43.077955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001477"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:43.271 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:43.450 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:43.451 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:43.463 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2026-02-22T20:46:43.077955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001477"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:43.464 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:43.464 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:43.465 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:43.465 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:43.465 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:43.680 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:43.681 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":820,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:43.693 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:43.694 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:43.870 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:43.871 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:43.882 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2026-02-22T20:46:43.077955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001477"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705831,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705830,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:46.884 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:47.090 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:47.090 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:47.103 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2026-02-22T20:46:47.078502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001488"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:47.103 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:47.282 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:47.282 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:47.295 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2026-02-22T20:46:47.078502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001488"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:47.296 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:47.297 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:47.297 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:47.297 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:47.297 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:47.509 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:47.509 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":824,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:47.521 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:47.522 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:47.698 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:47.698 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:47.710 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2026-02-22T20:46:47.078502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001488"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:50.711 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:50.898 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:50.898 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:50.911 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2026-02-22T20:46:49.078726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001524"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705832,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705831,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:50.912 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:51.093 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:51.093 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:51.106 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2026-02-22T20:46:51.078971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001531"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:51.107 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:51.107 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:51.108 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:51.108 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:51.108 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:51.324 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:51.324 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":828,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:51.337 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:51.337 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:51.515 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:51.515 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:51.527 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2026-02-22T20:46:51.078971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001531"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:54.529 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:54.713 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:54.714 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:54.726 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2026-02-22T20:46:53.079195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001513"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:54.727 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:54.905 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:54.906 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:54.917 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2026-02-22T20:46:53.079195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001513"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705833,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705832,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:54.919 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:54.919 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:54.919 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:54.919 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:54.920 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:55.137 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:55.137 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":831,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:55.151 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:55.151 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:55.329 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:55.329 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:55.341 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2026-02-22T20:46:55.079448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001493"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:58.342 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:58.521 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:58.522 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:58.535 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2026-02-22T20:46:57.079688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001475"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:58.536 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:58.717 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:58.717 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:58.729 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2026-02-22T20:46:57.079688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001475"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:46:58.730 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:46:58.730 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:46:58.731 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:46:58.731 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:46:58.731 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:46:58.942 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:58.942 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":835,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:46:58.955 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:46:58.955 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:46:59.135 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:46:59.136 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:46:59.148 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2026-02-22T20:46:59.079940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001438"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705834,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705833,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:02.150 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:02.340 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:02.341 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:02.353 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2026-02-22T20:47:01.080127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:02.355 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:02.537 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:02.538 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:02.549 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2026-02-22T20:47:01.080127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:02.551 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:02.551 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:02.551 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:02.551 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:02.552 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:02.770 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:02.770 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":839,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:02.782 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:02.783 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:02.959 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:02.960 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:02.971 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2026-02-22T20:47:01.080127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705835,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705834,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:05.974 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:06.152 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:06.153 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:06.165 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2026-02-22T20:47:05.080517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:06.167 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:06.342 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:06.342 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:06.354 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2026-02-22T20:47:05.080517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:06.355 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:06.355 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:06.356 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:06.356 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:06.356 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:06.572 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:06.573 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":843,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:06.586 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:06.587 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:06.763 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:06.763 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:06.774 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2026-02-22T20:47:05.080517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001321"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:09.776 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:09.957 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:09.957 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:09.969 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2026-02-22T20:47:09.080948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001259"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:09.971 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:10.147 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:10.147 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:10.159 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2026-02-22T20:47:09.080948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001259"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:10.160 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:10.160 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:10.160 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:10.160 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:10.161 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:10.380 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:10.380 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":847,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:10.393 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:10.393 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:10.571 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:10.571 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:10.584 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2026-02-22T20:47:09.080948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001259"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705836,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705835,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:13.586 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:13.762 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:13.762 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:13.774 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2026-02-22T20:47:13.081529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705838,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:13.775 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:13.955 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:13.955 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:13.967 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2026-02-22T20:47:13.081529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705838,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:13.968 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:13.968 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:13.969 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:13.969 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:13.969 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:14.184 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:14.185 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":850,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:14.199 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:14.199 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:14.377 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:14.377 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:14.389 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2026-02-22T20:47:13.081529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001402"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705838,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705837,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705836,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:17.392 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:17.570 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:17.571 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:17.582 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2026-02-22T20:47:17.081922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001405"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705839,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705838,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705838,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:17.583 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:17.762 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:17.763 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:17.775 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2026-02-22T20:47:17.081922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001405"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705839,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705838,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705838,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:17.776 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:17.777 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:17.777 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:17.777 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:17.777 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:17.993 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:17.993 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":854,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:18.006 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:18.006 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:18.182 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:18.183 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:18.194 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2026-02-22T20:47:17.081922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001405"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705839,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705838,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705837,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705838,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:21.197 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:21.375 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:21.376 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:21.387 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2026-02-22T20:47:21.082381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705838,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:21.389 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:21.569 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:21.569 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:21.581 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2026-02-22T20:47:21.082381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705838,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:21.583 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:21.583 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:21.583 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:21.583 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:21.584 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:21.796 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:21.796 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":858,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:21.809 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:21.809 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:21.990 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:21.991 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:22.003 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2026-02-22T20:47:21.082381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001432"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:25.038223+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:43:25.038223+0000","last_peered":"2026-02-22T20:43:25.038223+0000","last_clean":"2026-02-22T20:43:25.038223+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:43:25.038223+0000","last_undegraded":"2026-02-22T20:43:25.038223+0000","last_fullsized":"2026-02-22T20:43:25.038223+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705839,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705838,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:25.004 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:25.184 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:25.185 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:25.197 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2026-02-22T20:47:25.082869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001339"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:25.198 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:25.374 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:25.374 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:25.386 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2026-02-22T20:47:25.082869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001339"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:25.387 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:25.388 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:25.388 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:25.388 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:25.388 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:25.606 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:25.606 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":862,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:25.619 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:25.620 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:25.799 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:25.799 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:25.811 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2026-02-22T20:47:25.082869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001339"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:28.813 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:28.992 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:28.992 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:29.004 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2026-02-22T20:47:27.083089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001362"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:29.006 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:29.183 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:29.183 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:29.194 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2026-02-22T20:47:29.083333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001411"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:29.195 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:29.195 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:29.195 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:29.195 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:29.195 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:29.412 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:29.412 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":866,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:29.424 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:29.425 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:29.601 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:29.601 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:29.613 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2026-02-22T20:47:29.083333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001411"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705840,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705839,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:32.616 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:32.795 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:32.795 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:32.807 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2026-02-22T20:47:31.083547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:32.807 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:32.985 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:32.985 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:32.997 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2026-02-22T20:47:31.083547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001415"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:32.998 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:32.998 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:32.998 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:32.998 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:32.998 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:33.212 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:33.212 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":869,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:33.226 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:33.226 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:33.404 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:33.404 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:33.416 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2026-02-22T20:47:33.083819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001437"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705841,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705840,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:36.417 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:36.593 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:36.594 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:36.605 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2026-02-22T20:47:35.084023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001403"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:36.606 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:36.781 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:36.782 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:36.793 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2026-02-22T20:47:35.084023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001403"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:36.795 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:36.795 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:36.795 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:36.795 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:36.796 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:37.011 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:37.011 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":873,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:37.024 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:37.024 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:37.202 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:37.203 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:37.214 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2026-02-22T20:47:37.084241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001372"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:40.217 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:40.396 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:40.396 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:40.408 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2026-02-22T20:47:39.084447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001357"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:40.409 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:40.586 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:40.586 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:40.598 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2026-02-22T20:47:39.084447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001357"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:40.598 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:40.598 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:40.599 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:40.599 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:40.599 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:40.814 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:40.814 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":877,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:40.827 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:40.827 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:41.002 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:41.003 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:41.014 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2026-02-22T20:47:39.084447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001357"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705842,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705841,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:44.017 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:44.199 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:44.199 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:44.211 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2026-02-22T20:47:43.084914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705844,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:44.212 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:44.388 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:44.389 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:44.401 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2026-02-22T20:47:43.084914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705844,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:44.402 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:44.402 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:44.403 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:44.403 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:44.403 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:44.618 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:44.619 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":881,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:44.631 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:44.631 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:44.811 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:44.812 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:44.823 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2026-02-22T20:47:43.084914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705844,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705843,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705842,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:47.825 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:48.005 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:48.006 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:48.017 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2026-02-22T20:47:47.085368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001345"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705845,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705844,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705844,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:48.019 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:48.197 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:48.198 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:48.210 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2026-02-22T20:47:47.085368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001345"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705845,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705844,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705844,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:48.211 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:48.212 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:48.212 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:48.212 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:48.212 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:48.425 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:48.425 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":885,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:48.438 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:48.438 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:48.621 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:48.622 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:48.633 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2026-02-22T20:47:47.085368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001345"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039462+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:43:40.039462+0000","last_peered":"2026-02-22T20:43:40.039462+0000","last_clean":"2026-02-22T20:43:40.039462+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:43:40.039462+0000","last_undegraded":"2026-02-22T20:43:40.039462+0000","last_fullsized":"2026-02-22T20:43:40.039462+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039685+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:43:40.039685+0000","last_peered":"2026-02-22T20:43:40.039685+0000","last_clean":"2026-02-22T20:43:40.039685+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:43:40.039685+0000","last_undegraded":"2026-02-22T20:43:40.039685+0000","last_fullsized":"2026-02-22T20:43:40.039685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:43:40.039666+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:43:40.039666+0000","last_peered":"2026-02-22T20:43:40.039666+0000","last_clean":"2026-02-22T20:43:40.039666+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:43:40.039666+0000","last_undegraded":"2026-02-22T20:43:40.039666+0000","last_fullsized":"2026-02-22T20:43:40.039666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705845,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705844,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705843,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705844,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:51.635 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:51.810 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:51.811 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:51.821 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2026-02-22T20:47:51.085759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001312"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705845,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705844,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:51.822 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:51.998 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:51.998 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:52.010 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2026-02-22T20:47:51.085759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001312"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705845,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705844,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:52.011 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:52.011 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 25, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039773+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:43:45.039773+0000', 'last_peered': '2026-02-22T20:43:45.039773+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:43:45.039773+0000', 'last_undegraded': '2026-02-22T20:43:45.039773+0000', 'last_fullsized': '2026-02-22T20:43:45.039773+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:52.012 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:52.012 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 18, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:43:45.039960+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:43:45.039960+0000', 'last_peered': '2026-02-22T20:43:45.039960+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:43:45.039960+0000', 'last_undegraded': '2026-02-22T20:43:45.039960+0000', 'last_fullsized': '2026-02-22T20:43:45.039960+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:52.012 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:52.227 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:52.227 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":888,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:52.241 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:52.241 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:52.420 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:52.420 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:52.432 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2026-02-22T20:47:51.085759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001312"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":25,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039773+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:43:45.039773+0000","last_peered":"2026-02-22T20:43:45.039773+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:43:45.039773+0000","last_undegraded":"2026-02-22T20:43:45.039773+0000","last_fullsized":"2026-02-22T20:43:45.039773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":18,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:43:45.039960+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:43:45.039960+0000","last_peered":"2026-02-22T20:43:45.039960+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:43:45.039960+0000","last_undegraded":"2026-02-22T20:43:45.039960+0000","last_fullsized":"2026-02-22T20:43:45.039960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705845,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705844,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:55.434 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:55.613 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:55.614 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:55.627 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2026-02-22T20:47:55.086249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001335"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:55.628 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:55.802 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:55.803 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:55.814 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2026-02-22T20:47:55.086249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001335"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:55.816 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:55.816 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:55.816 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:55.816 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:55.817 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:56.030 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:56.031 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":892,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:56.042 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:56.043 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:56.222 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:56.222 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:56.234 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2026-02-22T20:47:55.086249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001335"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:59.236 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:59.416 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:59.417 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:59.428 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2026-02-22T20:47:59.086733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001364"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:59.430 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:47:59.615 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:59.615 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:47:59.627 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2026-02-22T20:47:59.086733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001364"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:47:59.629 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:47:59.629 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:47:59.629 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:47:59.629 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:47:59.630 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:47:59.843 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:47:59.843 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":896,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:47:59.854 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:47:59.855 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:00.034 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:00.034 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:00.046 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2026-02-22T20:47:59.086733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001364"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705846,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705845,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:03.047 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:03.226 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:03.227 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:03.238 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2026-02-22T20:48:03.087243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001483"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:03.240 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:03.421 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:03.422 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:03.433 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2026-02-22T20:48:03.087243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001483"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:03.435 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:03.435 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:03.435 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:03.435 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:03.436 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:03.658 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:03.658 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":900,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:03.671 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:03.671 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:03.852 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:03.852 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:03.863 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2026-02-22T20:48:03.087243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001483"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705847,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705846,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:06.865 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:07.045 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:07.046 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:07.057 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2026-02-22T20:48:05.087457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001455"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:07.060 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:07.237 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:07.238 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:07.250 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2026-02-22T20:48:07.087724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001474"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:07.250 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:07.250 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:07.250 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:07.250 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:07.251 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:07.464 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:07.464 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":904,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:07.477 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:07.477 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:07.656 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:07.657 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:07.669 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2026-02-22T20:48:07.087724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001474"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:10.670 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:10.853 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:10.853 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:10.866 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2026-02-22T20:48:09.087939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001481"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:10.867 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:11.044 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:11.044 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:11.056 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2026-02-22T20:48:09.087939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001481"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705848,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705847,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:11.056 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:11.056 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:11.056 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:11.056 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:11.057 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:11.272 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:11.273 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":908,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:11.286 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:11.287 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:11.464 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:11.464 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:11.476 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2026-02-22T20:48:11.088200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001467"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:14.478 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:14.660 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:14.660 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:14.673 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2026-02-22T20:48:13.088417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001441"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:14.674 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:14.852 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:14.852 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:14.865 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2026-02-22T20:48:13.088417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001441"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705849,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705848,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:14.866 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:14.866 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:14.866 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:14.866 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:14.866 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:15.082 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:15.082 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":911,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:15.095 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:15.095 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:15.271 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:15.272 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:15.284 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2026-02-22T20:48:15.088683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001440"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:18.285 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:18.469 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:18.470 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:18.482 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2026-02-22T20:48:17.088908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001451"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:18.483 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:18.664 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:18.665 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:18.676 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2026-02-22T20:48:17.088908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001451"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:18.677 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:18.677 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:18.677 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:18.678 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:18.678 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:18.893 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:18.893 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":915,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:18.906 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:18.907 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:19.086 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:19.086 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:19.098 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2026-02-22T20:48:17.088908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001451"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705850,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705849,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:22.100 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:22.283 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:22.283 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:22.295 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2026-02-22T20:48:21.089331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:22.297 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:22.476 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:22.477 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:22.489 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2026-02-22T20:48:21.089331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:22.490 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:22.490 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:22.491 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:22.491 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:22.491 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:22.705 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:22.706 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":919,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:22.718 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:22.718 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:22.895 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:22.896 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:22.907 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2026-02-22T20:48:21.089331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":40,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:05.041233+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:44:05.041233+0000","last_peered":"2026-02-22T20:44:05.041233+0000","last_clean":"2026-02-22T20:44:05.041233+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:44:05.041233+0000","last_undegraded":"2026-02-22T20:44:05.041233+0000","last_fullsized":"2026-02-22T20:44:05.041233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705851,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705850,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:25.909 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:26.086 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:26.086 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:26.097 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2026-02-22T20:48:25.089841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001424"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:26.099 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:26.275 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:26.276 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:26.288 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2026-02-22T20:48:25.089841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001424"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:26.290 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:26.290 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:26.290 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:26.290 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:26.291 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:26.502 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:26.502 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":923,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:26.515 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:26.515 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:26.697 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:26.697 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:26.709 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2026-02-22T20:48:25.089841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001424"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:29.710 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:29.889 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:29.890 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:29.903 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2026-02-22T20:48:29.090344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001435"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:29.904 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:30.081 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:30.082 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:30.094 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2026-02-22T20:48:29.090344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001435"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:30.095 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:30.095 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:30.096 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:30.096 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:30.096 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:30.311 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:30.311 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":927,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:30.324 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:30.325 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:30.504 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:30.504 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:30.516 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2026-02-22T20:48:29.090344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001435"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:44:10.041759+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:44:10.041759+0000","last_peered":"2026-02-22T20:44:10.041759+0000","last_clean":"2026-02-22T20:44:10.041759+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:44:10.041759+0000","last_undegraded":"2026-02-22T20:44:10.041759+0000","last_fullsized":"2026-02-22T20:44:10.041759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705851,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:33.519 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:33.697 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:33.698 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:33.710 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2026-02-22T20:48:33.090901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001569"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705853,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:33.711 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:33.886 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:33.886 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:33.898 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2026-02-22T20:48:33.090901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001569"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705853,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:33.899 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:33.899 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:33.900 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:33.900 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:33.900 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:34.113 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:34.113 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":930,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:34.126 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:34.126 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:34.303 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:34.304 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:34.315 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2026-02-22T20:48:33.090901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001569"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705852,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705853,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:37.316 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:37.497 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:37.497 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:37.510 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2026-02-22T20:48:37.091429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001587"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705853,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:37.510 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:37.686 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:37.686 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:37.697 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2026-02-22T20:48:37.091429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001587"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705853,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:37.698 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:37.698 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:37.698 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:37.698 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:37.699 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:37.911 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:37.911 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":934,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:37.925 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:37.926 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:38.106 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:38.107 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:38.119 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2026-02-22T20:48:37.091429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001587"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705853,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:41.121 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:41.300 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:41.300 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:41.312 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2026-02-22T20:48:41.091878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001534"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:41.313 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:41.491 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:41.491 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:41.503 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2026-02-22T20:48:41.091878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001534"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:41.505 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:41.505 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:41.505 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:41.505 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:41.506 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:41.722 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:41.722 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":938,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:41.735 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:41.735 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:41.909 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:41.910 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:41.921 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2026-02-22T20:48:41.091878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001534"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705854,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:44.923 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:45.106 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:45.106 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:45.119 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2026-02-22T20:48:45.092358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:45.120 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:45.295 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:45.296 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:45.308 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2026-02-22T20:48:45.092358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:45.309 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:45.309 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:45.309 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:45.310 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:45.310 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:45.528 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:45.528 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":942,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:45.540 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:45.540 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:45.718 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:45.718 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:45.729 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2026-02-22T20:48:45.092358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001457"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:48.732 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:48.913 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:48.913 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:48.925 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2026-02-22T20:48:47.092579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001421"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:48.927 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:49.108 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:49.108 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:49.120 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2026-02-22T20:48:49.092843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001414"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:49.121 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:49.121 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:49.122 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:49.122 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:49.122 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:49.338 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:49.338 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":946,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:49.351 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:49.351 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:49.527 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:49.528 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:49.540 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2026-02-22T20:48:49.092843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001414"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705855,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:52.540 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:52.720 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:52.721 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:52.732 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2026-02-22T20:48:51.093053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001424"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:52.734 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:52.913 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:52.914 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:52.925 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2026-02-22T20:48:51.093053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001424"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:52.927 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:52.927 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:52.927 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:52.927 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:52.928 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:53.144 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:53.145 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":949,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:53.158 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:53.158 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:53.336 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:53.336 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:53.348 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2026-02-22T20:48:53.093309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001430"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705856,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:56.351 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:56.530 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:56.530 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:56.541 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2026-02-22T20:48:55.093518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001418"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:56.543 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:56.720 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:56.721 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:56.733 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2026-02-22T20:48:55.093518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001418"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:48:56.734 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:48:56.734 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:48:56.735 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:48:56.735 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:48:56.735 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:48:56.949 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:56.949 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":953,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:48:56.962 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:48:56.962 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:48:57.139 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:48:57.140 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:48:57.151 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2026-02-22T20:48:57.093804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001446"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:00.153 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:00.334 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:00.335 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:00.347 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2026-02-22T20:48:59.094021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001442"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:00.349 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:00.528 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:00.529 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:00.540 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2026-02-22T20:48:59.094021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001442"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:00.540 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:00.541 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:00.541 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:00.541 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:00.541 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:00.753 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:00.753 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":957,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:00.765 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:00.766 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:00.943 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:00.944 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:00.955 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2026-02-22T20:48:59.094021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001442"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:03.957 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:04.136 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:04.136 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:04.148 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2026-02-22T20:49:03.094479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001426"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705859,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:04.149 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:04.328 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:04.328 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:04.340 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2026-02-22T20:49:03.094479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001426"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705859,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:04.340 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:04.341 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:04.341 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:04.341 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:04.341 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:04.557 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:04.557 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":961,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-22T20:34:10.990919+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:04.570 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:04.570 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:04.748 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:04.749 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:04.760 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2026-02-22T20:49:03.094479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001426"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705858,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705859,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:07.762 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:07.942 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:07.942 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:07.955 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2026-02-22T20:49:07.095011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705859,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:07.957 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:08.134 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:08.134 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:08.147 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2026-02-22T20:49:07.095011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705859,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:08.148 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:08.149 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:08.149 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:08.149 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:08.149 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:08.362 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:08.362 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":965,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":4,"modified":"2026-02-22T20:49:07.095034+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2026-02-22T20:49:08.375 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:08.375 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:08.554 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:08.555 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:08.567 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2026-02-22T20:49:07.095011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001492"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705859,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:11.569 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:11.750 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:11.750 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:11.762 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2026-02-22T20:49:11.095423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:11.764 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:11.942 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:11.942 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:11.955 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2026-02-22T20:49:11.095423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:11.956 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:11.956 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:11.956 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:11.956 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:11.956 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:12.168 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:12.168 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":968,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:12.180 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:12.180 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:12.360 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:12.361 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:12.374 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2026-02-22T20:49:11.095423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001401"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705860,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:15.375 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:15.552 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:15.552 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:15.564 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2026-02-22T20:49:15.095938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001458"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:15.564 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:15.742 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:15.743 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:15.755 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2026-02-22T20:49:15.095938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001458"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:15.756 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:15.756 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:15.757 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:15.757 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:15.757 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:15.974 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:15.974 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":972,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:15.987 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:15.988 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:16.166 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:16.167 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:16.178 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2026-02-22T20:49:15.095938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001458"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:19.180 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:19.360 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:19.361 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:19.372 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2026-02-22T20:49:19.096412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:19.373 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:19.551 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:19.551 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:19.563 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2026-02-22T20:49:19.096412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:19.563 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:19.563 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:19.563 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:19.563 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:19.564 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:19.779 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:19.779 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":976,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:19.790 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:19.790 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:19.966 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:19.967 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:19.979 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2026-02-22T20:49:19.096412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705861,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:22.981 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:23.160 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:23.160 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:23.172 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2026-02-22T20:49:23.096939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001515"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:23.173 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:23.349 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:23.349 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:23.361 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2026-02-22T20:49:23.096939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001515"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:23.361 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:23.361 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:23.362 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:23.362 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:23.362 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:23.577 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:23.578 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":980,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:23.590 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:23.591 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:23.768 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:23.768 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:23.780 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2026-02-22T20:49:23.096939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001515"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705863,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705862,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:26.782 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:26.961 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:26.961 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:26.973 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2026-02-22T20:49:25.097174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001498"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:26.974 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:27.154 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:27.155 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:27.166 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2026-02-22T20:49:27.097454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001516"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:27.168 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:27.168 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:27.168 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:27.168 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:27.169 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:27.386 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:27.386 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":984,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:27.399 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:27.399 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:27.573 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:27.573 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:27.584 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2026-02-22T20:49:27.097454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001516"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:30.586 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:30.765 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:30.766 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:30.778 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2026-02-22T20:49:29.097679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001529"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:30.779 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:30.955 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:30.956 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:30.968 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2026-02-22T20:49:29.097679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001529"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:25.047233+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:45:25.047233+0000","last_peered":"2026-02-22T20:45:25.047233+0000","last_clean":"2026-02-22T20:45:25.047233+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:45:25.047233+0000","last_undegraded":"2026-02-22T20:45:25.047233+0000","last_fullsized":"2026-02-22T20:45:25.047233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705864,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705863,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:30.969 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:30.969 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:30.969 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:30.969 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:30.970 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:31.183 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:31.183 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":987,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:31.196 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:31.196 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:31.375 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:31.376 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:31.388 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2026-02-22T20:49:31.097923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001511"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:34.390 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:34.566 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:34.566 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:34.578 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2026-02-22T20:49:33.098145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001482"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:34.580 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:34.760 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:34.760 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:34.772 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2026-02-22T20:49:33.098145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001482"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705864,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:34.774 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:34.774 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:34.774 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:34.774 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:34.775 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:34.993 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:34.994 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":991,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:35.007 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:35.007 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:35.185 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:35.185 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:35.198 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2026-02-22T20:49:35.098326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001387"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:38.200 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:38.378 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:38.378 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:38.391 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2026-02-22T20:49:37.098567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:38.392 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:38.570 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:38.571 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:38.583 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2026-02-22T20:49:37.098567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:38.584 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:38.584 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:38.585 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:38.585 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:38.585 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:38.800 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:38.800 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":995,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:38.813 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:38.813 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:38.990 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:38.991 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:39.002 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2026-02-22T20:49:37.098567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001392"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705866,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705865,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:42.003 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:42.182 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:42.182 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:42.195 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2026-02-22T20:49:41.099009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001330"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:42.196 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:42.373 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:42.373 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:42.385 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2026-02-22T20:49:41.099009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001330"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:42.386 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:42.386 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:42.386 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:42.386 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:42.386 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:42.600 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:42.601 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":999,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:42.614 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:42.614 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:42.793 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:42.794 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:42.805 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2026-02-22T20:49:41.099009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001330"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:45.808 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:45.986 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:45.986 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:45.997 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2026-02-22T20:49:45.099553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001407"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:45.998 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:46.176 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:46.177 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:46.188 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2026-02-22T20:49:45.099553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001407"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:46.189 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:46.189 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:46.190 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:46.190 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:46.190 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:46.405 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:46.405 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1003,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:46.418 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:46.418 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:46.593 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:46.593 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:46.604 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2026-02-22T20:49:45.099553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001407"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:49.606 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:49.785 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:49.786 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:49.797 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2026-02-22T20:49:49.100083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001515"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:49.798 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:49.973 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:49.973 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:49.985 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2026-02-22T20:49:49.100083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001515"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:49.986 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:49.986 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:49.987 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:49.987 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:49.987 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:50.202 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:50.202 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1006,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:50.215 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:50.215 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:50.392 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:50.393 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:50.404 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2026-02-22T20:49:49.100083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001515"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705867,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:53.406 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:53.587 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:53.588 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:53.599 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2026-02-22T20:49:53.100624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001615"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:53.601 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:53.779 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:53.779 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:53.790 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2026-02-22T20:49:53.100624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001615"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:53.791 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:53.791 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:53.791 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:53.791 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:53.791 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:54.008 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:54.008 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1010,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:54.021 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:54.021 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:54.203 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:54.204 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:54.216 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2026-02-22T20:49:53.100624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001615"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048835+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:45:45.048835+0000","last_peered":"2026-02-22T20:45:45.048835+0000","last_clean":"2026-02-22T20:45:45.048835+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:45:45.048835+0000","last_undegraded":"2026-02-22T20:45:45.048835+0000","last_fullsized":"2026-02-22T20:45:45.048835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048824+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:45:45.048824+0000","last_peered":"2026-02-22T20:45:45.048824+0000","last_clean":"2026-02-22T20:45:45.048824+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:45:45.048824+0000","last_undegraded":"2026-02-22T20:45:45.048824+0000","last_fullsized":"2026-02-22T20:45:45.048824+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705869,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705868,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:57.219 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:57.401 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:57.401 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:57.414 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2026-02-22T20:49:57.101085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001532"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.059011+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:47:50.059011+0000","last_peered":"2026-02-22T20:47:50.059011+0000","last_clean":"2026-02-22T20:47:50.059011+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:47:50.059011+0000","last_undegraded":"2026-02-22T20:47:50.059011+0000","last_fullsized":"2026-02-22T20:47:50.059011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058983+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:47:50.058983+0000","last_peered":"2026-02-22T20:47:50.058983+0000","last_clean":"2026-02-22T20:47:50.058983+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:47:50.058983+0000","last_undegraded":"2026-02-22T20:47:50.058983+0000","last_fullsized":"2026-02-22T20:47:50.058983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:57.415 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:57.591 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:57.591 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:57.603 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2026-02-22T20:49:57.101085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001532"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.059011+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:47:50.059011+0000","last_peered":"2026-02-22T20:47:50.059011+0000","last_clean":"2026-02-22T20:47:50.059011+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:47:50.059011+0000","last_undegraded":"2026-02-22T20:47:50.059011+0000","last_fullsized":"2026-02-22T20:47:50.059011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058983+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:47:50.058983+0000","last_peered":"2026-02-22T20:47:50.058983+0000","last_clean":"2026-02-22T20:47:50.058983+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:47:50.058983+0000","last_undegraded":"2026-02-22T20:47:50.058983+0000","last_fullsized":"2026-02-22T20:47:50.058983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:49:57.603 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:49:57.604 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 26, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.048812+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:45:50.048812+0000', 'last_peered': '2026-02-22T20:45:50.048812+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:45:50.048812+0000', 'last_undegraded': '2026-02-22T20:45:50.048812+0000', 'last_fullsized': '2026-02-22T20:45:50.048812+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:49:57.604 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:49:57.604 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 19, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:45:50.049101+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:45:50.049101+0000', 'last_peered': '2026-02-22T20:45:50.049101+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:45:50.049101+0000', 'last_undegraded': '2026-02-22T20:45:50.049101+0000', 'last_fullsized': '2026-02-22T20:45:50.049101+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:49:57.604 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:49:57.817 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:57.817 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1014,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:49:57.830 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:49:57.831 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:49:58.008 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:49:58.008 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:49:58.020 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2026-02-22T20:49:57.101085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001532"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":35,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:45:45.048512+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:45:45.048512+0000","last_peered":"2026-02-22T20:45:45.048512+0000","last_clean":"2026-02-22T20:45:45.048512+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:45:45.048512+0000","last_undegraded":"2026-02-22T20:45:45.048512+0000","last_fullsized":"2026-02-22T20:45:45.048512+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.059011+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:47:50.059011+0000","last_peered":"2026-02-22T20:47:50.059011+0000","last_clean":"2026-02-22T20:47:50.059011+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:47:50.059011+0000","last_undegraded":"2026-02-22T20:47:50.059011+0000","last_fullsized":"2026-02-22T20:47:50.059011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":26,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.048812+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:45:50.048812+0000","last_peered":"2026-02-22T20:45:50.048812+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:45:50.048812+0000","last_undegraded":"2026-02-22T20:45:50.048812+0000","last_fullsized":"2026-02-22T20:45:50.048812+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":19,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:45:50.049101+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:45:50.049101+0000","last_peered":"2026-02-22T20:45:50.049101+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:45:50.049101+0000","last_undegraded":"2026-02-22T20:45:50.049101+0000","last_fullsized":"2026-02-22T20:45:50.049101+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058983+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:47:50.058983+0000","last_peered":"2026-02-22T20:47:50.058983+0000","last_clean":"2026-02-22T20:47:50.058983+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:47:50.058983+0000","last_undegraded":"2026-02-22T20:47:50.058983+0000","last_fullsized":"2026-02-22T20:47:50.058983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705870,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705869,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:01.021 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:50:01.200 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:01.201 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:50:01.213 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2026-02-22T20:50:01.101470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001387"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058989+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:47:50.058989+0000","last_peered":"2026-02-22T20:47:50.058989+0000","last_clean":"2026-02-22T20:47:50.058989+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:47:50.058989+0000","last_undegraded":"2026-02-22T20:47:50.058989+0000","last_fullsized":"2026-02-22T20:47:50.058989+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.059011+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:47:50.059011+0000","last_peered":"2026-02-22T20:47:50.059011+0000","last_clean":"2026-02-22T20:47:50.059011+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:47:50.059011+0000","last_undegraded":"2026-02-22T20:47:50.059011+0000","last_fullsized":"2026-02-22T20:47:50.059011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":27,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:47:55.059464+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:47:55.059464+0000","last_peered":"2026-02-22T20:47:55.059464+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:47:55.059464+0000","last_undegraded":"2026-02-22T20:47:55.059464+0000","last_fullsized":"2026-02-22T20:47:55.059464+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:47:55.059455+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:47:55.059455+0000","last_peered":"2026-02-22T20:47:55.059455+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:47:55.059455+0000","last_undegraded":"2026-02-22T20:47:55.059455+0000","last_fullsized":"2026-02-22T20:47:55.059455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058983+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:47:50.058983+0000","last_peered":"2026-02-22T20:47:50.058983+0000","last_clean":"2026-02-22T20:47:50.058983+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:47:50.058983+0000","last_undegraded":"2026-02-22T20:47:50.058983+0000","last_fullsized":"2026-02-22T20:47:50.058983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:01.215 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:50:01.414 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:01.414 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:50:01.426 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2026-02-22T20:50:01.101470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001387"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058989+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:47:50.058989+0000","last_peered":"2026-02-22T20:47:50.058989+0000","last_clean":"2026-02-22T20:47:50.058989+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:47:50.058989+0000","last_undegraded":"2026-02-22T20:47:50.058989+0000","last_fullsized":"2026-02-22T20:47:50.058989+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.059011+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:47:50.059011+0000","last_peered":"2026-02-22T20:47:50.059011+0000","last_clean":"2026-02-22T20:47:50.059011+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:47:50.059011+0000","last_undegraded":"2026-02-22T20:47:50.059011+0000","last_fullsized":"2026-02-22T20:47:50.059011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":27,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:47:55.059464+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:47:55.059464+0000","last_peered":"2026-02-22T20:47:55.059464+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:47:55.059464+0000","last_undegraded":"2026-02-22T20:47:55.059464+0000","last_fullsized":"2026-02-22T20:47:55.059464+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:47:55.059455+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:47:55.059455+0000","last_peered":"2026-02-22T20:47:55.059455+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:47:55.059455+0000","last_undegraded":"2026-02-22T20:47:55.059455+0000","last_fullsized":"2026-02-22T20:47:55.059455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058983+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:47:50.058983+0000","last_peered":"2026-02-22T20:47:50.058983+0000","last_clean":"2026-02-22T20:47:50.058983+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:47:50.058983+0000","last_undegraded":"2026-02-22T20:47:50.058983+0000","last_fullsized":"2026-02-22T20:47:50.058983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:01.427 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:50:01.427 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "11'1", 'reported_seq': 27, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:47:55.059464+0000', 'last_change': '2026-02-22T20:33:33.143372+0000', 'last_active': '2026-02-22T20:47:55.059464+0000', 'last_peered': '2026-02-22T20:47:55.059464+0000', 'last_clean': '2026-02-22T20:33:14.007552+0000', 'last_became_active': '2026-02-22T20:33:22.074095+0000', 'last_became_peered': '2026-02-22T20:33:22.074095+0000', 'last_unstale': '2026-02-22T20:47:55.059464+0000', 'last_undegraded': '2026-02-22T20:47:55.059464+0000', 'last_fullsized': '2026-02-22T20:47:55.059464+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '1'], 'object_location_counts': [{'shards': '1,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:50:01.428 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:50:01.428 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "13'2", 'reported_seq': 20, 'reported_epoch': 17, 'state': 'active+recovering', 'last_fresh': '2026-02-22T20:47:55.059455+0000', 'last_change': '2026-02-22T20:33:33.143053+0000', 'last_active': '2026-02-22T20:47:55.059455+0000', 'last_peered': '2026-02-22T20:47:55.059455+0000', 'last_clean': '2026-02-22T20:33:18.852969+0000', 'last_became_active': '2026-02-22T20:33:22.074403+0000', 'last_became_peered': '2026-02-22T20:33:22.074403+0000', 'last_unstale': '2026-02-22T20:47:55.059455+0000', 'last_undegraded': '2026-02-22T20:47:55.059455+0000', 'last_fullsized': '2026-02-22T20:47:55.059455+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 1, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 2, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': ['0', '1'], 'object_location_counts': [{'shards': '0,1', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:50:01.428 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:50:01.643 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:01.644 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1018,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+recovering","count":2}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55508992,"bytes_avail":193218019328,"bytes_total":193273528320,"misplaced_objects":3,"misplaced_total":4,"misplaced_ratio":0.75},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:50:01.658 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:50:01.658 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:50:01.836 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:01.836 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:50:01.848 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2026-02-22T20:50:01.101470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001387"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058989+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:47:50.058989+0000","last_peered":"2026-02-22T20:47:50.058989+0000","last_clean":"2026-02-22T20:47:50.058989+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:47:50.058989+0000","last_undegraded":"2026-02-22T20:47:50.058989+0000","last_fullsized":"2026-02-22T20:47:50.058989+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.059011+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:47:50.059011+0000","last_peered":"2026-02-22T20:47:50.059011+0000","last_clean":"2026-02-22T20:47:50.059011+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:47:50.059011+0000","last_undegraded":"2026-02-22T20:47:50.059011+0000","last_fullsized":"2026-02-22T20:47:50.059011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":27,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:47:55.059464+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:47:55.059464+0000","last_peered":"2026-02-22T20:47:55.059464+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:47:55.059464+0000","last_undegraded":"2026-02-22T20:47:55.059464+0000","last_fullsized":"2026-02-22T20:47:55.059464+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:47:55.059455+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:47:55.059455+0000","last_peered":"2026-02-22T20:47:55.059455+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:47:55.059455+0000","last_undegraded":"2026-02-22T20:47:55.059455+0000","last_fullsized":"2026-02-22T20:47:55.059455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058983+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:47:50.058983+0000","last_peered":"2026-02-22T20:47:50.058983+0000","last_clean":"2026-02-22T20:47:50.058983+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:47:50.058983+0000","last_undegraded":"2026-02-22T20:47:50.058983+0000","last_fullsized":"2026-02-22T20:47:50.058983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:04.849 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:50:05.029 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:05.030 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:50:05.042 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2026-02-22T20:50:03.101701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54208,"kb_used_data":288,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689472,"statfs":{"total":193273528320,"available":193218019328,"internally_reserved":0,"allocated":294912,"data_stored":89440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001373"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058989+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:47:50.058989+0000","last_peered":"2026-02-22T20:47:50.058989+0000","last_clean":"2026-02-22T20:47:50.058989+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:47:50.058989+0000","last_undegraded":"2026-02-22T20:47:50.058989+0000","last_fullsized":"2026-02-22T20:47:50.058989+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.059011+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:47:50.059011+0000","last_peered":"2026-02-22T20:47:50.059011+0000","last_clean":"2026-02-22T20:47:50.059011+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:47:50.059011+0000","last_undegraded":"2026-02-22T20:47:50.059011+0000","last_fullsized":"2026-02-22T20:47:50.059011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":27,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:47:55.059464+0000","last_change":"2026-02-22T20:33:33.143372+0000","last_active":"2026-02-22T20:47:55.059464+0000","last_peered":"2026-02-22T20:47:55.059464+0000","last_clean":"2026-02-22T20:33:14.007552+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:47:55.059464+0000","last_undegraded":"2026-02-22T20:47:55.059464+0000","last_fullsized":"2026-02-22T20:47:55.059464+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","1"],"object_location_counts":[{"shards":"1,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":20,"reported_epoch":17,"state":"active+recovering","last_fresh":"2026-02-22T20:47:55.059455+0000","last_change":"2026-02-22T20:33:33.143053+0000","last_active":"2026-02-22T20:47:55.059455+0000","last_peered":"2026-02-22T20:47:55.059455+0000","last_clean":"2026-02-22T20:33:18.852969+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:47:55.059455+0000","last_undegraded":"2026-02-22T20:47:55.059455+0000","last_fullsized":"2026-02-22T20:47:55.059455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058983+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:47:50.058983+0000","last_peered":"2026-02-22T20:47:50.058983+0000","last_clean":"2026-02-22T20:47:50.058983+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:47:50.058983+0000","last_undegraded":"2026-02-22T20:47:50.058983+0000","last_fullsized":"2026-02-22T20:47:50.058983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27104,"kb_used_data":144,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344736,"statfs":{"total":96636764160,"available":96609009664,"internally_reserved":0,"allocated":147456,"data_stored":44720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705870,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:05.043 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:50:05.222 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:05.223 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:50:05.235 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2026-02-22T20:50:05.101905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54216,"kb_used_data":296,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689464,"statfs":{"total":193273528320,"available":193218011136,"internally_reserved":0,"allocated":303104,"data_stored":89478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":-1,"num_objects_missing":-1,"num_objects_degraded":0,"num_objects_misplaced":-3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001281"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058989+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:47:50.058989+0000","last_peered":"2026-02-22T20:47:50.058989+0000","last_clean":"2026-02-22T20:47:50.058989+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:47:50.058989+0000","last_undegraded":"2026-02-22T20:47:50.058989+0000","last_fullsized":"2026-02-22T20:47:50.058989+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.059011+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:47:50.059011+0000","last_peered":"2026-02-22T20:47:50.059011+0000","last_clean":"2026-02-22T20:47:50.059011+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:47:50.059011+0000","last_undegraded":"2026-02-22T20:47:50.059011+0000","last_fullsized":"2026-02-22T20:47:50.059011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:50:00.686277+0000","last_change":"2026-02-22T20:50:00.686229+0000","last_active":"2026-02-22T20:50:00.686277+0000","last_peered":"2026-02-22T20:50:00.686277+0000","last_clean":"2026-02-22T20:50:00.686277+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:50:00.686277+0000","last_undegraded":"2026-02-22T20:50:00.686277+0000","last_fullsized":"2026-02-22T20:50:00.686277+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:06:06.953930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":27,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:50:01.425469+0000","last_change":"2026-02-22T20:50:01.425426+0000","last_active":"2026-02-22T20:50:01.425469+0000","last_peered":"2026-02-22T20:50:01.425469+0000","last_clean":"2026-02-22T20:50:01.425469+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:50:01.425469+0000","last_undegraded":"2026-02-22T20:50:01.425469+0000","last_fullsized":"2026-02-22T20:50:01.425469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:35:50.034064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058983+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:47:50.058983+0000","last_peered":"2026-02-22T20:47:50.058983+0000","last_clean":"2026-02-22T20:47:50.058983+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:47:50.058983+0000","last_undegraded":"2026-02-22T20:47:50.058983+0000","last_fullsized":"2026-02-22T20:47:50.058983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27108,"kb_used_data":148,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344732,"statfs":{"total":96636764160,"available":96609005568,"internally_reserved":0,"allocated":151552,"data_stored":44739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27108,"kb_used_data":148,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344732,"statfs":{"total":96636764160,"available":96609005568,"internally_reserved":0,"allocated":151552,"data_stored":44739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705872,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:05.237 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:50:05.453 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:05.454 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1022,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":17,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771792401,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":8}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":55517184,"bytes_avail":193218011136,"bytes_total":193273528320,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":1,"recovering_keys_per_sec":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:50:05.466 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:50:05.466 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:50:05.644 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:05.644 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:50:05.656 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2026-02-22T20:50:05.101905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54216,"kb_used_data":296,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689464,"statfs":{"total":193273528320,"available":193218011136,"internally_reserved":0,"allocated":303104,"data_stored":89478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":-1,"num_objects_missing":-1,"num_objects_degraded":0,"num_objects_misplaced":-3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001281"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058989+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:47:50.058989+0000","last_peered":"2026-02-22T20:47:50.058989+0000","last_clean":"2026-02-22T20:47:50.058989+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:47:50.058989+0000","last_undegraded":"2026-02-22T20:47:50.058989+0000","last_fullsized":"2026-02-22T20:47:50.058989+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.059011+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:47:50.059011+0000","last_peered":"2026-02-22T20:47:50.059011+0000","last_clean":"2026-02-22T20:47:50.059011+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:47:50.059011+0000","last_undegraded":"2026-02-22T20:47:50.059011+0000","last_fullsized":"2026-02-22T20:47:50.059011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:50:00.686277+0000","last_change":"2026-02-22T20:50:00.686229+0000","last_active":"2026-02-22T20:50:00.686277+0000","last_peered":"2026-02-22T20:50:00.686277+0000","last_clean":"2026-02-22T20:50:00.686277+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:50:00.686277+0000","last_undegraded":"2026-02-22T20:50:00.686277+0000","last_fullsized":"2026-02-22T20:50:00.686277+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:06:06.953930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":27,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:50:01.425469+0000","last_change":"2026-02-22T20:50:01.425426+0000","last_active":"2026-02-22T20:50:01.425469+0000","last_peered":"2026-02-22T20:50:01.425469+0000","last_clean":"2026-02-22T20:50:01.425469+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:50:01.425469+0000","last_undegraded":"2026-02-22T20:50:01.425469+0000","last_fullsized":"2026-02-22T20:50:01.425469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:35:50.034064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058983+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:47:50.058983+0000","last_peered":"2026-02-22T20:47:50.058983+0000","last_clean":"2026-02-22T20:47:50.058983+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:47:50.058983+0000","last_undegraded":"2026-02-22T20:47:50.058983+0000","last_fullsized":"2026-02-22T20:47:50.058983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27108,"kb_used_data":148,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344732,"statfs":{"total":96636764160,"available":96609005568,"internally_reserved":0,"allocated":151552,"data_stored":44739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27108,"kb_used_data":148,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344732,"statfs":{"total":96636764160,"available":96609005568,"internally_reserved":0,"allocated":151552,"data_stored":44739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705872,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:08.657 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:50:08.839 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:08.839 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:50:08.851 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2026-02-22T20:50:07.102137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54216,"kb_used_data":296,"kb_used_omap":18,"kb_used_meta":53869,"kb_avail":188689464,"statfs":{"total":193273528320,"available":193218011136,"internally_reserved":0,"allocated":303104,"data_stored":89478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19002,"internal_metadata":55162310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":-1,"num_objects_missing":-1,"num_objects_degraded":0,"num_objects_misplaced":-3,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001294"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:20.051675+0000","last_change":"2026-02-22T20:33:22.074235+0000","last_active":"2026-02-22T20:46:20.051675+0000","last_peered":"2026-02-22T20:46:20.051675+0000","last_clean":"2026-02-22T20:46:20.051675+0000","last_became_active":"2026-02-22T20:33:22.073932+0000","last_became_peered":"2026-02-22T20:33:22.073932+0000","last_unstale":"2026-02-22T20:46:20.051675+0000","last_undegraded":"2026-02-22T20:46:20.051675+0000","last_fullsized":"2026-02-22T20:46:20.051675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:05:14.618728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:46:15.051203+0000","last_change":"2026-02-22T20:33:22.074437+0000","last_active":"2026-02-22T20:46:15.051203+0000","last_peered":"2026-02-22T20:46:15.051203+0000","last_clean":"2026-02-22T20:46:15.051203+0000","last_became_active":"2026-02-22T20:33:22.074050+0000","last_became_peered":"2026-02-22T20:33:22.074050+0000","last_unstale":"2026-02-22T20:46:15.051203+0000","last_undegraded":"2026-02-22T20:46:15.051203+0000","last_fullsized":"2026-02-22T20:46:15.051203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:53.196293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058989+0000","last_change":"2026-02-22T20:33:16.183569+0000","last_active":"2026-02-22T20:47:50.058989+0000","last_peered":"2026-02-22T20:47:50.058989+0000","last_clean":"2026-02-22T20:47:50.058989+0000","last_became_active":"2026-02-22T20:33:14.007050+0000","last_became_peered":"2026-02-22T20:33:14.007050+0000","last_unstale":"2026-02-22T20:47:50.058989+0000","last_undegraded":"2026-02-22T20:47:50.058989+0000","last_fullsized":"2026-02-22T20:47:50.058989+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:39:53.237600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.059011+0000","last_change":"2026-02-22T20:33:22.074593+0000","last_active":"2026-02-22T20:47:50.059011+0000","last_peered":"2026-02-22T20:47:50.059011+0000","last_clean":"2026-02-22T20:47:50.059011+0000","last_became_active":"2026-02-22T20:33:22.074238+0000","last_became_peered":"2026-02-22T20:33:22.074238+0000","last_unstale":"2026-02-22T20:47:50.059011+0000","last_undegraded":"2026-02-22T20:47:50.059011+0000","last_fullsized":"2026-02-22T20:47:50.059011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:19:55.242533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"11'1","reported_seq":33,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:50:00.686277+0000","last_change":"2026-02-22T20:50:00.686229+0000","last_active":"2026-02-22T20:50:00.686277+0000","last_peered":"2026-02-22T20:50:00.686277+0000","last_clean":"2026-02-22T20:50:00.686277+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:50:00.686277+0000","last_undegraded":"2026-02-22T20:50:00.686277+0000","last_fullsized":"2026-02-22T20:50:00.686277+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:06:06.953930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"13'2","reported_seq":27,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:50:01.425469+0000","last_change":"2026-02-22T20:50:01.425426+0000","last_active":"2026-02-22T20:50:01.425469+0000","last_peered":"2026-02-22T20:50:01.425469+0000","last_clean":"2026-02-22T20:50:01.425469+0000","last_became_active":"2026-02-22T20:33:22.074403+0000","last_became_peered":"2026-02-22T20:33:22.074403+0000","last_unstale":"2026-02-22T20:50:01.425469+0000","last_undegraded":"2026-02-22T20:50:01.425469+0000","last_fullsized":"2026-02-22T20:50:01.425469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:35:50.034064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:50.058983+0000","last_change":"2026-02-22T20:33:22.074155+0000","last_active":"2026-02-22T20:47:50.058983+0000","last_peered":"2026-02-22T20:47:50.058983+0000","last_clean":"2026-02-22T20:47:50.058983+0000","last_became_active":"2026-02-22T20:33:22.073911+0000","last_became_peered":"2026-02-22T20:33:22.073911+0000","last_unstale":"2026-02-22T20:47:50.058983+0000","last_undegraded":"2026-02-22T20:47:50.058983+0000","last_fullsized":"2026-02-22T20:47:50.058983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:40:32.600134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-22T20:47:25.056696+0000","last_change":"2026-02-22T20:33:22.073842+0000","last_active":"2026-02-22T20:47:25.056696+0000","last_peered":"2026-02-22T20:47:25.056696+0000","last_clean":"2026-02-22T20:47:25.056696+0000","last_became_active":"2026-02-22T20:33:22.073595+0000","last_became_peered":"2026-02-22T20:33:22.073595+0000","last_unstale":"2026-02-22T20:47:25.056696+0000","last_undegraded":"2026-02-22T20:47:25.056696+0000","last_fullsized":"2026-02-22T20:47:25.056696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:57:23.769190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27108,"kb_used_data":148,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344732,"statfs":{"total":96636764160,"available":96609005568,"internally_reserved":0,"allocated":151552,"data_stored":44739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9500,"internal_metadata":27581156},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27108,"kb_used_data":148,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344732,"statfs":{"total":96636764160,"available":96609005568,"internally_reserved":0,"allocated":151552,"data_stored":44739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9502,"internal_metadata":27581154},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":38654705871,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":38654705872,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:08.853 INFO:tasks.osd_recovery.ceph_manager:clean! 2026-02-22T20:50:08.853 INFO:tasks.osd_recovery:rados -p rbd bench 20 write -b 1 --no-cleanup 2026-02-22T20:50:08.854 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd bench 20 write -b 1 --no-cleanup 2026-02-22T20:50:08.892 INFO:teuthology.orchestra.run.trial084.stdout:hints = 1 2026-02-22T20:50:08.892 INFO:teuthology.orchestra.run.trial084.stdout:Maintaining 16 concurrent writes of 1 bytes to objects of size 1 for up to 20 seconds or 0 objects 2026-02-22T20:50:08.893 INFO:teuthology.orchestra.run.trial084.stdout:Object prefix: benchmark_data_trial084_98775 2026-02-22T20:50:08.893 INFO:teuthology.orchestra.run.trial084.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T20:50:08.893 INFO:teuthology.orchestra.run.trial084.stdout: 0 0 0 0 0 0 - 0 2026-02-22T20:50:09.892 INFO:teuthology.orchestra.run.trial084.stdout: 1 16 7445 7429 0.00708434 0.00708485 0.0020664 0.00214871 2026-02-22T20:50:10.893 INFO:teuthology.orchestra.run.trial084.stdout: 2 16 14344 14328 0.00683163 0.0065794 0.00183674 0.00223038 2026-02-22T20:50:11.892 INFO:teuthology.orchestra.run.trial084.stdout: 3 16 20616 20600 0.00654809 0.00598145 0.00227083 0.00232706 2026-02-22T20:50:12.892 INFO:teuthology.orchestra.run.trial084.stdout: 4 16 26421 26405 0.00629499 0.00553608 0.00454418 0.00242095 2026-02-22T20:50:13.892 INFO:teuthology.orchestra.run.trial084.stdout: 5 16 31675 31659 0.00603803 0.0050106 0.00255205 0.00252412 2026-02-22T20:50:14.893 INFO:teuthology.orchestra.run.trial084.stdout: 6 16 36510 36494 0.00580013 0.00461102 0.00254074 0.00262797 2026-02-22T20:50:15.892 INFO:teuthology.orchestra.run.trial084.stdout: 7 16 41087 41071 0.00559507 0.00436497 0.00396836 0.00272455 2026-02-22T20:50:16.893 INFO:teuthology.orchestra.run.trial084.stdout: 8 16 45395 45379 0.0054092 0.00410843 0.00370093 0.00281833 2026-02-22T20:50:17.893 INFO:teuthology.orchestra.run.trial084.stdout: 9 15 49424 49409 0.00523518 0.00384331 0.0054118 0.00291208 2026-02-22T20:50:18.893 INFO:teuthology.orchestra.run.trial084.stdout: 10 15 53214 53199 0.00507308 0.00361443 0.00337484 0.00300493 2026-02-22T20:50:19.893 INFO:teuthology.orchestra.run.trial084.stdout: 11 16 56712 56696 0.00491505 0.003335 0.003233 0.00310212 2026-02-22T20:50:20.893 INFO:teuthology.orchestra.run.trial084.stdout: 12 16 60071 60055 0.00477239 0.00320339 0.00734074 0.00319451 2026-02-22T20:50:21.893 INFO:teuthology.orchestra.run.trial084.stdout: 13 16 63214 63198 0.00463584 0.0029974 0.00612568 0.0032889 2026-02-22T20:50:22.893 INFO:teuthology.orchestra.run.trial084.stdout: 14 16 66229 66213 0.00451007 0.00287533 0.00249169 0.00338047 2026-02-22T20:50:23.893 INFO:teuthology.orchestra.run.trial084.stdout: 15 16 69080 69064 0.00439065 0.00271893 0.00187105 0.00347197 2026-02-22T20:50:24.893 INFO:teuthology.orchestra.run.trial084.stdout: 16 16 71832 71816 0.00428025 0.00262451 0.0111954 0.00356215 2026-02-22T20:50:25.893 INFO:teuthology.orchestra.run.trial084.stdout: 17 16 74445 74429 0.00417505 0.00249195 0.00219753 0.00365232 2026-02-22T20:50:26.893 INFO:teuthology.orchestra.run.trial084.stdout: 18 16 76962 76946 0.00407645 0.0024004 0.0040491 0.00374041 2026-02-22T20:50:27.894 INFO:teuthology.orchestra.run.trial084.stdout: 19 15 79400 79385 0.00398429 0.00232601 0.00772977 0.00382717 2026-02-22T20:50:28.894 INFO:teuthology.orchestra.run.trial084.stdout:2026-02-22T20:50:28.892894+0000 min lat: 0.00106945 max lat: 0.0194378 avg lat: 0.0039134 2026-02-22T20:50:28.894 INFO:teuthology.orchestra.run.trial084.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T20:50:28.894 INFO:teuthology.orchestra.run.trial084.stdout: 20 11 81732 81721 0.00389645 0.00222778 0.00718694 0.0039134 2026-02-22T20:50:29.894 INFO:teuthology.orchestra.run.trial084.stdout:Total time run: 20.0078 2026-02-22T20:50:29.895 INFO:teuthology.orchestra.run.trial084.stdout:Total writes made: 81732 2026-02-22T20:50:29.895 INFO:teuthology.orchestra.run.trial084.stdout:Write size: 1 2026-02-22T20:50:29.895 INFO:teuthology.orchestra.run.trial084.stdout:Object size: 1 2026-02-22T20:50:29.895 INFO:teuthology.orchestra.run.trial084.stdout:Bandwidth (MB/sec): 0.00389577 2026-02-22T20:50:29.895 INFO:teuthology.orchestra.run.trial084.stdout:Stddev Bandwidth: 0.00147988 2026-02-22T20:50:29.895 INFO:teuthology.orchestra.run.trial084.stdout:Max bandwidth (MB/sec): 0.00708485 2026-02-22T20:50:29.895 INFO:teuthology.orchestra.run.trial084.stdout:Min bandwidth (MB/sec): 0.00222778 2026-02-22T20:50:29.895 INFO:teuthology.orchestra.run.trial084.stdout:Average IOPS: 4085 2026-02-22T20:50:29.895 INFO:teuthology.orchestra.run.trial084.stdout:Stddev IOPS: 1551.77 2026-02-22T20:50:29.895 INFO:teuthology.orchestra.run.trial084.stdout:Max IOPS: 7429 2026-02-22T20:50:29.895 INFO:teuthology.orchestra.run.trial084.stdout:Min IOPS: 2336 2026-02-22T20:50:29.896 INFO:teuthology.orchestra.run.trial084.stdout:Average Latency(s): 0.00391397 2026-02-22T20:50:29.896 INFO:teuthology.orchestra.run.trial084.stdout:Stddev Latency(s): 0.00215457 2026-02-22T20:50:29.896 INFO:teuthology.orchestra.run.trial084.stdout:Max latency(s): 0.0194378 2026-02-22T20:50:29.896 INFO:teuthology.orchestra.run.trial084.stdout:Min latency(s): 0.00106945 2026-02-22T20:50:29.903 INFO:tasks.osd_recovery:rados -p rbd put foo.1 /etc/passwd 2026-02-22T20:50:29.903 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.1 /etc/passwd 2026-02-22T20:50:29.985 INFO:tasks.osd_recovery:rados -p rbd put foo.2 /etc/passwd 2026-02-22T20:50:29.986 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.2 /etc/passwd 2026-02-22T20:50:30.067 INFO:tasks.osd_recovery:rados -p rbd put foo.3 /etc/passwd 2026-02-22T20:50:30.067 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.3 /etc/passwd 2026-02-22T20:50:30.149 INFO:tasks.osd_recovery:rados -p rbd put foo.4 /etc/passwd 2026-02-22T20:50:30.149 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.4 /etc/passwd 2026-02-22T20:50:30.232 INFO:tasks.osd_recovery:rados -p rbd put foo.5 /etc/passwd 2026-02-22T20:50:30.232 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.5 /etc/passwd 2026-02-22T20:50:30.317 INFO:tasks.osd_recovery:rados -p rbd put foo.6 /etc/passwd 2026-02-22T20:50:30.317 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.6 /etc/passwd 2026-02-22T20:50:30.399 INFO:tasks.osd_recovery:rados -p rbd put foo.7 /etc/passwd 2026-02-22T20:50:30.399 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.7 /etc/passwd 2026-02-22T20:50:30.480 INFO:tasks.osd_recovery:rados -p rbd put foo.8 /etc/passwd 2026-02-22T20:50:30.480 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.8 /etc/passwd 2026-02-22T20:50:30.563 INFO:tasks.osd_recovery:rados -p rbd put foo.9 /etc/passwd 2026-02-22T20:50:30.563 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.9 /etc/passwd 2026-02-22T20:50:30.646 INFO:tasks.osd_recovery:rados -p rbd put foo.10 /etc/passwd 2026-02-22T20:50:30.647 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.10 /etc/passwd 2026-02-22T20:50:30.729 INFO:tasks.osd_recovery:rados -p rbd put foo.11 /etc/passwd 2026-02-22T20:50:30.729 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.11 /etc/passwd 2026-02-22T20:50:30.811 INFO:tasks.osd_recovery:rados -p rbd put foo.12 /etc/passwd 2026-02-22T20:50:30.811 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.12 /etc/passwd 2026-02-22T20:50:30.893 INFO:tasks.osd_recovery:rados -p rbd put foo.13 /etc/passwd 2026-02-22T20:50:30.893 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.13 /etc/passwd 2026-02-22T20:50:30.976 INFO:tasks.osd_recovery:rados -p rbd put foo.14 /etc/passwd 2026-02-22T20:50:30.976 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.14 /etc/passwd 2026-02-22T20:50:31.057 INFO:tasks.osd_recovery:rados -p rbd put foo.15 /etc/passwd 2026-02-22T20:50:31.058 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.15 /etc/passwd 2026-02-22T20:50:31.139 INFO:tasks.osd_recovery:rados -p rbd put foo.16 /etc/passwd 2026-02-22T20:50:31.139 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.16 /etc/passwd 2026-02-22T20:50:31.221 INFO:tasks.osd_recovery:rados -p rbd put foo.17 /etc/passwd 2026-02-22T20:50:31.221 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.17 /etc/passwd 2026-02-22T20:50:31.303 INFO:tasks.osd_recovery:rados -p rbd put foo.18 /etc/passwd 2026-02-22T20:50:31.303 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.18 /etc/passwd 2026-02-22T20:50:31.384 INFO:tasks.osd_recovery:rados -p rbd put foo.19 /etc/passwd 2026-02-22T20:50:31.384 DEBUG:teuthology.orchestra.run.trial084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p rbd put foo.19 /etc/passwd 2026-02-22T20:50:31.466 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 1 2026-02-22T20:50:31.691 INFO:tasks.ceph.mon.a.trial084.stderr:2026-02-22T20:50:31.689+0000 7fb50627a6c0 -1 mon.a@0(leader).osd e17 definitely_dead 0 2026-02-22T20:50:32.198 INFO:teuthology.orchestra.run.trial084.stderr:marked in osd.0. marked in osd.1. 2026-02-22T20:50:32.214 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 3 2026-02-22T20:50:32.399 INFO:tasks.ceph.mon.a.trial084.stderr:2026-02-22T20:50:32.398+0000 7fb50627a6c0 -1 mon.a@0(leader).osd e18 definitely_dead 0 2026-02-22T20:50:33.132 INFO:teuthology.orchestra.run.trial084.stderr:marked out osd.2. marked out osd.3. 2026-02-22T20:50:43.149 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-22T20:50:43.150 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-22T20:50:43.150 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-22T20:50:43.151 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-22T20:50:43.265 INFO:teuthology.orchestra.run.trial084.stdout:38654705879 2026-02-22T20:50:43.266 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T20:50:43.267 INFO:teuthology.orchestra.run.trial084.stdout:38654705881 2026-02-22T20:50:43.268 DEBUG:teuthology.parallel:result is None 2026-02-22T20:50:43.268 INFO:teuthology.orchestra.run.trial084.stdout:38654705880 2026-02-22T20:50:43.269 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T20:50:43.270 INFO:teuthology.orchestra.run.trial084.stdout:38654705880 2026-02-22T20:50:43.270 DEBUG:teuthology.parallel:result is None 2026-02-22T20:50:43.455 INFO:teuthology.orchestra.run.trial084.stdout:38654705878 2026-02-22T20:50:43.469 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705879 got 38654705878 for osd.1 2026-02-22T20:50:43.508 INFO:teuthology.orchestra.run.trial084.stdout:38654705879 2026-02-22T20:50:43.522 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705880 got 38654705879 for osd.0 2026-02-22T20:50:44.470 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T20:50:44.523 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T20:50:44.655 INFO:teuthology.orchestra.run.trial084.stdout:38654705878 2026-02-22T20:50:44.669 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705879 got 38654705878 for osd.1 2026-02-22T20:50:44.704 INFO:teuthology.orchestra.run.trial084.stdout:38654705879 2026-02-22T20:50:44.718 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705880 got 38654705879 for osd.0 2026-02-22T20:50:45.669 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T20:50:45.719 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T20:50:45.855 INFO:teuthology.orchestra.run.trial084.stdout:38654705880 2026-02-22T20:50:45.870 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705879 got 38654705880 for osd.1 2026-02-22T20:50:45.871 DEBUG:teuthology.parallel:result is None 2026-02-22T20:50:45.903 INFO:teuthology.orchestra.run.trial084.stdout:38654705881 2026-02-22T20:50:45.919 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705880 got 38654705881 for osd.0 2026-02-22T20:50:45.919 DEBUG:teuthology.parallel:result is None 2026-02-22T20:50:55.920 INFO:tasks.osd_recovery.ceph_manager:waiting for peering to complete 2026-02-22T20:50:55.920 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:50:56.107 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:56.108 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:50:56.122 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2026-02-22T20:50:55.108266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":160444,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":102464,"kb_used_data":12448,"kb_used_omap":23,"kb_used_meta":89960,"kb_avail":188641216,"statfs":{"total":193273528320,"available":193168605184,"internally_reserved":0,"allocated":12746752,"data_stored":120598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24236,"internal_metadata":92119380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2240,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1120,"num_bytes_recovered":1120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001456"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10233,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131834+0000","last_change":"2026-02-22T20:50:34.374535+0000","last_active":"2026-02-22T20:50:35.131834+0000","last_peered":"2026-02-22T20:50:35.131834+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:50:34.369203+0000","last_became_peered":"2026-02-22T20:50:34.369203+0000","last_unstale":"2026-02-22T20:50:35.131834+0000","last_undegraded":"2026-02-22T20:50:35.131834+0000","last_fullsized":"2026-02-22T20:50:35.131834+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10475,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130756+0000","last_change":"2026-02-22T20:50:34.412693+0000","last_active":"2026-02-22T20:50:35.130756+0000","last_peered":"2026-02-22T20:50:35.130756+0000","last_clean":"2026-02-22T20:50:30.889887+0000","last_became_active":"2026-02-22T20:50:34.335145+0000","last_became_peered":"2026-02-22T20:50:34.335145+0000","last_unstale":"2026-02-22T20:50:35.130756+0000","last_undegraded":"2026-02-22T20:50:35.130756+0000","last_fullsized":"2026-02-22T20:50:35.130756+0000","mapping_epoch":19,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":34,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10148,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.237095+0000","last_change":"2026-02-22T20:50:35.237225+0000","last_active":"2026-02-22T20:50:35.237095+0000","last_peered":"2026-02-22T20:50:35.237095+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:50:35.232433+0000","last_became_peered":"2026-02-22T20:50:35.232433+0000","last_unstale":"2026-02-22T20:50:35.237095+0000","last_undegraded":"2026-02-22T20:50:35.237095+0000","last_fullsized":"2026-02-22T20:50:35.237095+0000","mapping_epoch":20,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10486,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130708+0000","last_change":"2026-02-22T20:50:34.368608+0000","last_active":"2026-02-22T20:50:35.130708+0000","last_peered":"2026-02-22T20:50:35.130708+0000","last_clean":"2026-02-22T20:50:31.300321+0000","last_became_active":"2026-02-22T20:50:34.362361+0000","last_became_peered":"2026-02-22T20:50:34.362361+0000","last_unstale":"2026-02-22T20:50:35.130708+0000","last_undegraded":"2026-02-22T20:50:35.130708+0000","last_fullsized":"2026-02-22T20:50:35.130708+0000","mapping_epoch":19,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":14595,"reported_epoch":21,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:50:55.020629+0000","last_change":"2026-02-22T20:50:34.426703+0000","last_active":"2026-02-22T20:50:55.020629+0000","last_peered":"2026-02-22T20:50:55.020629+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:50:34.404753+0000","last_became_peered":"2026-02-22T20:50:34.404753+0000","last_unstale":"2026-02-22T20:50:55.020629+0000","last_undegraded":"2026-02-22T20:50:55.020629+0000","last_fullsized":"2026-02-22T20:50:34.129679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17112,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1500,"num_bytes_recovered":1498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1499},{"shards":"2,3","objects":8556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10206,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131737+0000","last_change":"2026-02-22T20:50:34.368334+0000","last_active":"2026-02-22T20:50:35.131737+0000","last_peered":"2026-02-22T20:50:35.131737+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:50:34.362168+0000","last_became_peered":"2026-02-22T20:50:34.362168+0000","last_unstale":"2026-02-22T20:50:35.131737+0000","last_undegraded":"2026-02-22T20:50:35.131737+0000","last_fullsized":"2026-02-22T20:50:35.131737+0000","mapping_epoch":19,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20336,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10293,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130683+0000","last_change":"2026-02-22T20:50:34.366383+0000","last_active":"2026-02-22T20:50:35.130683+0000","last_peered":"2026-02-22T20:50:35.130683+0000","last_clean":"2026-02-22T20:50:30.477943+0000","last_became_active":"2026-02-22T20:50:34.356915+0000","last_became_peered":"2026-02-22T20:50:34.356915+0000","last_unstale":"2026-02-22T20:50:35.130683+0000","last_undegraded":"2026-02-22T20:50:35.130683+0000","last_fullsized":"2026-02-22T20:50:35.130683+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10271,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131745+0000","last_change":"2026-02-22T20:50:34.375783+0000","last_active":"2026-02-22T20:50:35.131745+0000","last_peered":"2026-02-22T20:50:35.131745+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:50:34.369192+0000","last_became_peered":"2026-02-22T20:50:34.369192+0000","last_unstale":"2026-02-22T20:50:35.131745+0000","last_undegraded":"2026-02-22T20:50:35.131745+0000","last_fullsized":"2026-02-22T20:50:35.131745+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":160444,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":682729472,"data_stored":747404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705884,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51300,"kb_used_data":6292,"kb_used_omap":11,"kb_used_meta":44980,"kb_avail":94320540,"statfs":{"total":96636764160,"available":96584232960,"internally_reserved":0,"allocated":6443008,"data_stored":60316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":46059690},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51164,"kb_used_data":6156,"kb_used_omap":11,"kb_used_meta":44980,"kb_avail":94320676,"statfs":{"total":96636764160,"available":96584372224,"internally_reserved":0,"allocated":6303744,"data_stored":60282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":46059690},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6270976,"data_stored":1531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6270976,"data_stored":1531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:56.123 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:50:56.308 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:56.308 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:50:56.322 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2026-02-22T20:50:55.108266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":160444,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":102464,"kb_used_data":12448,"kb_used_omap":23,"kb_used_meta":89960,"kb_avail":188641216,"statfs":{"total":193273528320,"available":193168605184,"internally_reserved":0,"allocated":12746752,"data_stored":120598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24236,"internal_metadata":92119380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2240,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1120,"num_bytes_recovered":1120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001456"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10233,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131834+0000","last_change":"2026-02-22T20:50:34.374535+0000","last_active":"2026-02-22T20:50:35.131834+0000","last_peered":"2026-02-22T20:50:35.131834+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:50:34.369203+0000","last_became_peered":"2026-02-22T20:50:34.369203+0000","last_unstale":"2026-02-22T20:50:35.131834+0000","last_undegraded":"2026-02-22T20:50:35.131834+0000","last_fullsized":"2026-02-22T20:50:35.131834+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10475,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130756+0000","last_change":"2026-02-22T20:50:34.412693+0000","last_active":"2026-02-22T20:50:35.130756+0000","last_peered":"2026-02-22T20:50:35.130756+0000","last_clean":"2026-02-22T20:50:30.889887+0000","last_became_active":"2026-02-22T20:50:34.335145+0000","last_became_peered":"2026-02-22T20:50:34.335145+0000","last_unstale":"2026-02-22T20:50:35.130756+0000","last_undegraded":"2026-02-22T20:50:35.130756+0000","last_fullsized":"2026-02-22T20:50:35.130756+0000","mapping_epoch":19,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":34,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10148,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.237095+0000","last_change":"2026-02-22T20:50:35.237225+0000","last_active":"2026-02-22T20:50:35.237095+0000","last_peered":"2026-02-22T20:50:35.237095+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:50:35.232433+0000","last_became_peered":"2026-02-22T20:50:35.232433+0000","last_unstale":"2026-02-22T20:50:35.237095+0000","last_undegraded":"2026-02-22T20:50:35.237095+0000","last_fullsized":"2026-02-22T20:50:35.237095+0000","mapping_epoch":20,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10486,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130708+0000","last_change":"2026-02-22T20:50:34.368608+0000","last_active":"2026-02-22T20:50:35.130708+0000","last_peered":"2026-02-22T20:50:35.130708+0000","last_clean":"2026-02-22T20:50:31.300321+0000","last_became_active":"2026-02-22T20:50:34.362361+0000","last_became_peered":"2026-02-22T20:50:34.362361+0000","last_unstale":"2026-02-22T20:50:35.130708+0000","last_undegraded":"2026-02-22T20:50:35.130708+0000","last_fullsized":"2026-02-22T20:50:35.130708+0000","mapping_epoch":19,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":14595,"reported_epoch":21,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:50:55.020629+0000","last_change":"2026-02-22T20:50:34.426703+0000","last_active":"2026-02-22T20:50:55.020629+0000","last_peered":"2026-02-22T20:50:55.020629+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:50:34.404753+0000","last_became_peered":"2026-02-22T20:50:34.404753+0000","last_unstale":"2026-02-22T20:50:55.020629+0000","last_undegraded":"2026-02-22T20:50:55.020629+0000","last_fullsized":"2026-02-22T20:50:34.129679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17112,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1500,"num_bytes_recovered":1498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1499},{"shards":"2,3","objects":8556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10206,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131737+0000","last_change":"2026-02-22T20:50:34.368334+0000","last_active":"2026-02-22T20:50:35.131737+0000","last_peered":"2026-02-22T20:50:35.131737+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:50:34.362168+0000","last_became_peered":"2026-02-22T20:50:34.362168+0000","last_unstale":"2026-02-22T20:50:35.131737+0000","last_undegraded":"2026-02-22T20:50:35.131737+0000","last_fullsized":"2026-02-22T20:50:35.131737+0000","mapping_epoch":19,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20336,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10293,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130683+0000","last_change":"2026-02-22T20:50:34.366383+0000","last_active":"2026-02-22T20:50:35.130683+0000","last_peered":"2026-02-22T20:50:35.130683+0000","last_clean":"2026-02-22T20:50:30.477943+0000","last_became_active":"2026-02-22T20:50:34.356915+0000","last_became_peered":"2026-02-22T20:50:34.356915+0000","last_unstale":"2026-02-22T20:50:35.130683+0000","last_undegraded":"2026-02-22T20:50:35.130683+0000","last_fullsized":"2026-02-22T20:50:35.130683+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10271,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131745+0000","last_change":"2026-02-22T20:50:34.375783+0000","last_active":"2026-02-22T20:50:35.131745+0000","last_peered":"2026-02-22T20:50:35.131745+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:50:34.369192+0000","last_became_peered":"2026-02-22T20:50:34.369192+0000","last_unstale":"2026-02-22T20:50:35.131745+0000","last_undegraded":"2026-02-22T20:50:35.131745+0000","last_fullsized":"2026-02-22T20:50:35.131745+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":160444,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":682729472,"data_stored":747404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705884,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51300,"kb_used_data":6292,"kb_used_omap":11,"kb_used_meta":44980,"kb_avail":94320540,"statfs":{"total":96636764160,"available":96584232960,"internally_reserved":0,"allocated":6443008,"data_stored":60316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":46059690},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51164,"kb_used_data":6156,"kb_used_omap":11,"kb_used_meta":44980,"kb_avail":94320676,"statfs":{"total":96636764160,"available":96584372224,"internally_reserved":0,"allocated":6303744,"data_stored":60282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":46059690},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6270976,"data_stored":1531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6270976,"data_stored":1531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:56.323 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:50:56.541 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:56.541 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1073,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":21,"num_osds":4,"num_up_osds":4,"osd_up_since":1771792388,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":104923136,"bytes_avail":193168605184,"bytes_total":193273528320,"misplaced_objects":160444,"misplaced_total":163508,"misplaced_ratio":0.98126085573794553,"recovering_objects_per_sec":93,"recovering_bytes_per_sec":93,"recovering_keys_per_sec":0,"num_objects_recovered":1120,"num_bytes_recovered":1120,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:50:56.555 INFO:tasks.osd_recovery.ceph_manager:{'fsid': 'a4ea559c-fa00-404b-b21c-f34ee9bd0795', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 1073, 'monmap': {'epoch': 1, 'min_mon_release_name': 'tentacle', 'num_mons': 3}, 'osdmap': {'epoch': 21, 'num_osds': 4, 'num_up_osds': 4, 'osd_up_since': 1771792388, 'num_in_osds': 2, 'osd_in_since': 1771793433, 'num_remapped_pgs': 8}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+remapped+backfill_wait', 'count': 7}, {'state_name': 'active+recovering+undersized+remapped', 'count': 1}], 'num_pgs': 8, 'num_pools': 1, 'num_objects': 81754, 'data_bytes': 372171, 'bytes_used': 104923136, 'bytes_avail': 193168605184, 'bytes_total': 193273528320, 'misplaced_objects': 160444, 'misplaced_total': 163508, 'misplaced_ratio': 0.9812608557379455, 'recovering_objects_per_sec': 93, 'recovering_bytes_per_sec': 93, 'recovering_keys_per_sec': 0, 'num_objects_recovered': 1120, 'num_bytes_recovered': 1120, 'num_keys_recovered': 0}, 'fsmap': {'epoch': 1, 'btime': '2026-02-22T20:33:03:266748+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs'], 'services': {}}, 'servicemap': {'epoch': 5, 'modified': '2026-02-22T20:49:11.095473+0000', 'services': {}}, 'progress_events': {}} 2026-02-22T20:50:56.556 INFO:tasks.osd_recovery.ceph_manager:active! 2026-02-22T20:50:56.556 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:50:56.738 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:56.739 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:50:56.751 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2026-02-22T20:50:55.108266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":160444,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":102464,"kb_used_data":12448,"kb_used_omap":23,"kb_used_meta":89960,"kb_avail":188641216,"statfs":{"total":193273528320,"available":193168605184,"internally_reserved":0,"allocated":12746752,"data_stored":120598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24236,"internal_metadata":92119380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2240,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1120,"num_bytes_recovered":1120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001456"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10233,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131834+0000","last_change":"2026-02-22T20:50:34.374535+0000","last_active":"2026-02-22T20:50:35.131834+0000","last_peered":"2026-02-22T20:50:35.131834+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:50:34.369203+0000","last_became_peered":"2026-02-22T20:50:34.369203+0000","last_unstale":"2026-02-22T20:50:35.131834+0000","last_undegraded":"2026-02-22T20:50:35.131834+0000","last_fullsized":"2026-02-22T20:50:35.131834+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10475,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130756+0000","last_change":"2026-02-22T20:50:34.412693+0000","last_active":"2026-02-22T20:50:35.130756+0000","last_peered":"2026-02-22T20:50:35.130756+0000","last_clean":"2026-02-22T20:50:30.889887+0000","last_became_active":"2026-02-22T20:50:34.335145+0000","last_became_peered":"2026-02-22T20:50:34.335145+0000","last_unstale":"2026-02-22T20:50:35.130756+0000","last_undegraded":"2026-02-22T20:50:35.130756+0000","last_fullsized":"2026-02-22T20:50:35.130756+0000","mapping_epoch":19,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":34,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10148,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.237095+0000","last_change":"2026-02-22T20:50:35.237225+0000","last_active":"2026-02-22T20:50:35.237095+0000","last_peered":"2026-02-22T20:50:35.237095+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:50:35.232433+0000","last_became_peered":"2026-02-22T20:50:35.232433+0000","last_unstale":"2026-02-22T20:50:35.237095+0000","last_undegraded":"2026-02-22T20:50:35.237095+0000","last_fullsized":"2026-02-22T20:50:35.237095+0000","mapping_epoch":20,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10486,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130708+0000","last_change":"2026-02-22T20:50:34.368608+0000","last_active":"2026-02-22T20:50:35.130708+0000","last_peered":"2026-02-22T20:50:35.130708+0000","last_clean":"2026-02-22T20:50:31.300321+0000","last_became_active":"2026-02-22T20:50:34.362361+0000","last_became_peered":"2026-02-22T20:50:34.362361+0000","last_unstale":"2026-02-22T20:50:35.130708+0000","last_undegraded":"2026-02-22T20:50:35.130708+0000","last_fullsized":"2026-02-22T20:50:35.130708+0000","mapping_epoch":19,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":14595,"reported_epoch":21,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:50:55.020629+0000","last_change":"2026-02-22T20:50:34.426703+0000","last_active":"2026-02-22T20:50:55.020629+0000","last_peered":"2026-02-22T20:50:55.020629+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:50:34.404753+0000","last_became_peered":"2026-02-22T20:50:34.404753+0000","last_unstale":"2026-02-22T20:50:55.020629+0000","last_undegraded":"2026-02-22T20:50:55.020629+0000","last_fullsized":"2026-02-22T20:50:34.129679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17112,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1500,"num_bytes_recovered":1498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1499},{"shards":"2,3","objects":8556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10206,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131737+0000","last_change":"2026-02-22T20:50:34.368334+0000","last_active":"2026-02-22T20:50:35.131737+0000","last_peered":"2026-02-22T20:50:35.131737+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:50:34.362168+0000","last_became_peered":"2026-02-22T20:50:34.362168+0000","last_unstale":"2026-02-22T20:50:35.131737+0000","last_undegraded":"2026-02-22T20:50:35.131737+0000","last_fullsized":"2026-02-22T20:50:35.131737+0000","mapping_epoch":19,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20336,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10293,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130683+0000","last_change":"2026-02-22T20:50:34.366383+0000","last_active":"2026-02-22T20:50:35.130683+0000","last_peered":"2026-02-22T20:50:35.130683+0000","last_clean":"2026-02-22T20:50:30.477943+0000","last_became_active":"2026-02-22T20:50:34.356915+0000","last_became_peered":"2026-02-22T20:50:34.356915+0000","last_unstale":"2026-02-22T20:50:35.130683+0000","last_undegraded":"2026-02-22T20:50:35.130683+0000","last_fullsized":"2026-02-22T20:50:35.130683+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10271,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131745+0000","last_change":"2026-02-22T20:50:34.375783+0000","last_active":"2026-02-22T20:50:35.131745+0000","last_peered":"2026-02-22T20:50:35.131745+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:50:34.369192+0000","last_became_peered":"2026-02-22T20:50:34.369192+0000","last_unstale":"2026-02-22T20:50:35.131745+0000","last_undegraded":"2026-02-22T20:50:35.131745+0000","last_fullsized":"2026-02-22T20:50:35.131745+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":160444,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":682729472,"data_stored":747404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705884,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51300,"kb_used_data":6292,"kb_used_omap":11,"kb_used_meta":44980,"kb_avail":94320540,"statfs":{"total":96636764160,"available":96584232960,"internally_reserved":0,"allocated":6443008,"data_stored":60316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":46059690},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51164,"kb_used_data":6156,"kb_used_omap":11,"kb_used_meta":44980,"kb_avail":94320676,"statfs":{"total":96636764160,"available":96584372224,"internally_reserved":0,"allocated":6303744,"data_stored":60282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":46059690},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6270976,"data_stored":1531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6270976,"data_stored":1531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:56.752 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:50:56.937 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:56.938 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:50:56.951 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":553,"stamp":"2026-02-22T20:50:55.108266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":160444,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":102464,"kb_used_data":12448,"kb_used_omap":23,"kb_used_meta":89960,"kb_avail":188641216,"statfs":{"total":193273528320,"available":193168605184,"internally_reserved":0,"allocated":12746752,"data_stored":120598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24236,"internal_metadata":92119380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2240,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1120,"num_bytes_recovered":1120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001456"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10233,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131834+0000","last_change":"2026-02-22T20:50:34.374535+0000","last_active":"2026-02-22T20:50:35.131834+0000","last_peered":"2026-02-22T20:50:35.131834+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:50:34.369203+0000","last_became_peered":"2026-02-22T20:50:34.369203+0000","last_unstale":"2026-02-22T20:50:35.131834+0000","last_undegraded":"2026-02-22T20:50:35.131834+0000","last_fullsized":"2026-02-22T20:50:35.131834+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10475,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130756+0000","last_change":"2026-02-22T20:50:34.412693+0000","last_active":"2026-02-22T20:50:35.130756+0000","last_peered":"2026-02-22T20:50:35.130756+0000","last_clean":"2026-02-22T20:50:30.889887+0000","last_became_active":"2026-02-22T20:50:34.335145+0000","last_became_peered":"2026-02-22T20:50:34.335145+0000","last_unstale":"2026-02-22T20:50:35.130756+0000","last_undegraded":"2026-02-22T20:50:35.130756+0000","last_fullsized":"2026-02-22T20:50:35.130756+0000","mapping_epoch":19,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":34,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10148,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.237095+0000","last_change":"2026-02-22T20:50:35.237225+0000","last_active":"2026-02-22T20:50:35.237095+0000","last_peered":"2026-02-22T20:50:35.237095+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:50:35.232433+0000","last_became_peered":"2026-02-22T20:50:35.232433+0000","last_unstale":"2026-02-22T20:50:35.237095+0000","last_undegraded":"2026-02-22T20:50:35.237095+0000","last_fullsized":"2026-02-22T20:50:35.237095+0000","mapping_epoch":20,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10486,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130708+0000","last_change":"2026-02-22T20:50:34.368608+0000","last_active":"2026-02-22T20:50:35.130708+0000","last_peered":"2026-02-22T20:50:35.130708+0000","last_clean":"2026-02-22T20:50:31.300321+0000","last_became_active":"2026-02-22T20:50:34.362361+0000","last_became_peered":"2026-02-22T20:50:34.362361+0000","last_unstale":"2026-02-22T20:50:35.130708+0000","last_undegraded":"2026-02-22T20:50:35.130708+0000","last_fullsized":"2026-02-22T20:50:35.130708+0000","mapping_epoch":19,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":14595,"reported_epoch":21,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:50:55.020629+0000","last_change":"2026-02-22T20:50:34.426703+0000","last_active":"2026-02-22T20:50:55.020629+0000","last_peered":"2026-02-22T20:50:55.020629+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:50:34.404753+0000","last_became_peered":"2026-02-22T20:50:34.404753+0000","last_unstale":"2026-02-22T20:50:55.020629+0000","last_undegraded":"2026-02-22T20:50:55.020629+0000","last_fullsized":"2026-02-22T20:50:34.129679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17112,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1500,"num_bytes_recovered":1498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1499},{"shards":"2,3","objects":8556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10206,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131737+0000","last_change":"2026-02-22T20:50:34.368334+0000","last_active":"2026-02-22T20:50:35.131737+0000","last_peered":"2026-02-22T20:50:35.131737+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:50:34.362168+0000","last_became_peered":"2026-02-22T20:50:34.362168+0000","last_unstale":"2026-02-22T20:50:35.131737+0000","last_undegraded":"2026-02-22T20:50:35.131737+0000","last_fullsized":"2026-02-22T20:50:35.131737+0000","mapping_epoch":19,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20336,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10293,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130683+0000","last_change":"2026-02-22T20:50:34.366383+0000","last_active":"2026-02-22T20:50:35.130683+0000","last_peered":"2026-02-22T20:50:35.130683+0000","last_clean":"2026-02-22T20:50:30.477943+0000","last_became_active":"2026-02-22T20:50:34.356915+0000","last_became_peered":"2026-02-22T20:50:34.356915+0000","last_unstale":"2026-02-22T20:50:35.130683+0000","last_undegraded":"2026-02-22T20:50:35.130683+0000","last_fullsized":"2026-02-22T20:50:35.130683+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10271,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131745+0000","last_change":"2026-02-22T20:50:34.375783+0000","last_active":"2026-02-22T20:50:35.131745+0000","last_peered":"2026-02-22T20:50:35.131745+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:50:34.369192+0000","last_became_peered":"2026-02-22T20:50:34.369192+0000","last_unstale":"2026-02-22T20:50:35.131745+0000","last_undegraded":"2026-02-22T20:50:35.131745+0000","last_fullsized":"2026-02-22T20:50:35.131745+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":160444,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":682729472,"data_stored":747404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705884,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51300,"kb_used_data":6292,"kb_used_omap":11,"kb_used_meta":44980,"kb_avail":94320540,"statfs":{"total":96636764160,"available":96584232960,"internally_reserved":0,"allocated":6443008,"data_stored":60316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":46059690},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51164,"kb_used_data":6156,"kb_used_omap":11,"kb_used_meta":44980,"kb_avail":94320676,"statfs":{"total":96636764160,"available":96584372224,"internally_reserved":0,"allocated":6303744,"data_stored":60282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":46059690},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6270976,"data_stored":1531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6270976,"data_stored":1531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:56.952 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:50:56.953 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10233, 'reported_epoch': 21, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:50:35.131834+0000', 'last_change': '2026-02-22T20:50:34.374535+0000', 'last_active': '2026-02-22T20:50:35.131834+0000', 'last_peered': '2026-02-22T20:50:35.131834+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:50:34.369203+0000', 'last_became_peered': '2026-02-22T20:50:34.369203+0000', 'last_unstale': '2026-02-22T20:50:35.131834+0000', 'last_undegraded': '2026-02-22T20:50:35.131834+0000', 'last_fullsized': '2026-02-22T20:50:35.131834+0000', 'mapping_epoch': 19, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:50:56.953 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:50:56.953 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10475, 'reported_epoch': 21, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:50:35.130756+0000', 'last_change': '2026-02-22T20:50:34.412693+0000', 'last_active': '2026-02-22T20:50:35.130756+0000', 'last_peered': '2026-02-22T20:50:35.130756+0000', 'last_clean': '2026-02-22T20:50:30.889887+0000', 'last_became_active': '2026-02-22T20:50:34.335145+0000', 'last_became_peered': '2026-02-22T20:50:34.335145+0000', 'last_unstale': '2026-02-22T20:50:35.130756+0000', 'last_undegraded': '2026-02-22T20:50:35.130756+0000', 'last_fullsized': '2026-02-22T20:50:35.130756+0000', 'mapping_epoch': 19, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20566, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 34, 'num_bytes_recovered': 34, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3, 2], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:50:56.953 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:50:56.953 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10148, 'reported_epoch': 21, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:50:35.237095+0000', 'last_change': '2026-02-22T20:50:35.237225+0000', 'last_active': '2026-02-22T20:50:35.237095+0000', 'last_peered': '2026-02-22T20:50:35.237095+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:50:35.232433+0000', 'last_became_peered': '2026-02-22T20:50:35.232433+0000', 'last_unstale': '2026-02-22T20:50:35.237095+0000', 'last_undegraded': '2026-02-22T20:50:35.237095+0000', 'last_fullsized': '2026-02-22T20:50:35.237095+0000', 'mapping_epoch': 20, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20202, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:50:56.953 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:50:56.953 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10486, 'reported_epoch': 21, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:50:35.130708+0000', 'last_change': '2026-02-22T20:50:34.368608+0000', 'last_active': '2026-02-22T20:50:35.130708+0000', 'last_peered': '2026-02-22T20:50:35.130708+0000', 'last_clean': '2026-02-22T20:50:31.300321+0000', 'last_became_active': '2026-02-22T20:50:34.362361+0000', 'last_became_peered': '2026-02-22T20:50:34.362361+0000', 'last_unstale': '2026-02-22T20:50:35.130708+0000', 'last_undegraded': '2026-02-22T20:50:35.130708+0000', 'last_fullsized': '2026-02-22T20:50:35.130708+0000', 'mapping_epoch': 19, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3, 2], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:50:56.953 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:50:56.953 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 14595, 'reported_epoch': 21, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-22T20:50:55.020629+0000', 'last_change': '2026-02-22T20:50:34.426703+0000', 'last_active': '2026-02-22T20:50:55.020629+0000', 'last_peered': '2026-02-22T20:50:55.020629+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:50:34.404753+0000', 'last_became_peered': '2026-02-22T20:50:34.404753+0000', 'last_unstale': '2026-02-22T20:50:55.020629+0000', 'last_undegraded': '2026-02-22T20:50:55.020629+0000', 'last_fullsized': '2026-02-22T20:50:34.129679+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17112, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1500, 'num_bytes_recovered': 1498, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 1499}, {'shards': '2,3', 'objects': 8556}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:50:56.953 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:50:56.953 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10206, 'reported_epoch': 21, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:50:35.131737+0000', 'last_change': '2026-02-22T20:50:34.368334+0000', 'last_active': '2026-02-22T20:50:35.131737+0000', 'last_peered': '2026-02-22T20:50:35.131737+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:50:34.362168+0000', 'last_became_peered': '2026-02-22T20:50:34.362168+0000', 'last_unstale': '2026-02-22T20:50:35.131737+0000', 'last_undegraded': '2026-02-22T20:50:35.131737+0000', 'last_fullsized': '2026-02-22T20:50:35.131737+0000', 'mapping_epoch': 19, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20336, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:50:56.954 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:50:56.954 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10293, 'reported_epoch': 21, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:50:35.130683+0000', 'last_change': '2026-02-22T20:50:34.366383+0000', 'last_active': '2026-02-22T20:50:35.130683+0000', 'last_peered': '2026-02-22T20:50:35.130683+0000', 'last_clean': '2026-02-22T20:50:30.477943+0000', 'last_became_active': '2026-02-22T20:50:34.356915+0000', 'last_became_peered': '2026-02-22T20:50:34.356915+0000', 'last_unstale': '2026-02-22T20:50:35.130683+0000', 'last_undegraded': '2026-02-22T20:50:35.130683+0000', 'last_fullsized': '2026-02-22T20:50:35.130683+0000', 'mapping_epoch': 19, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [3, 2], 'avail_no_missing': ['3', '2'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 3, 'purged_snaps': []} 2026-02-22T20:50:56.954 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:50:56.954 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10271, 'reported_epoch': 21, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:50:35.131745+0000', 'last_change': '2026-02-22T20:50:34.375783+0000', 'last_active': '2026-02-22T20:50:35.131745+0000', 'last_peered': '2026-02-22T20:50:35.131745+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:50:34.369192+0000', 'last_became_peered': '2026-02-22T20:50:34.369192+0000', 'last_unstale': '2026-02-22T20:50:35.131745+0000', 'last_undegraded': '2026-02-22T20:50:35.131745+0000', 'last_fullsized': '2026-02-22T20:50:35.131745+0000', 'mapping_epoch': 19, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:50:56.954 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:50:57.137 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:50:57.138 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:50:57.152 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2026-02-22T20:50:57.108528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":160444,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":102464,"kb_used_data":12448,"kb_used_omap":23,"kb_used_meta":89960,"kb_avail":188641216,"statfs":{"total":193273528320,"available":193168605184,"internally_reserved":0,"allocated":12746752,"data_stored":120598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24236,"internal_metadata":92119380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1534,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":767,"num_bytes_recovered":767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001494"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10233,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131834+0000","last_change":"2026-02-22T20:50:34.374535+0000","last_active":"2026-02-22T20:50:35.131834+0000","last_peered":"2026-02-22T20:50:35.131834+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:50:34.369203+0000","last_became_peered":"2026-02-22T20:50:34.369203+0000","last_unstale":"2026-02-22T20:50:35.131834+0000","last_undegraded":"2026-02-22T20:50:35.131834+0000","last_fullsized":"2026-02-22T20:50:35.131834+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10475,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130756+0000","last_change":"2026-02-22T20:50:34.412693+0000","last_active":"2026-02-22T20:50:35.130756+0000","last_peered":"2026-02-22T20:50:35.130756+0000","last_clean":"2026-02-22T20:50:30.889887+0000","last_became_active":"2026-02-22T20:50:34.335145+0000","last_became_peered":"2026-02-22T20:50:34.335145+0000","last_unstale":"2026-02-22T20:50:35.130756+0000","last_undegraded":"2026-02-22T20:50:35.130756+0000","last_fullsized":"2026-02-22T20:50:35.130756+0000","mapping_epoch":19,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":34,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10148,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.237095+0000","last_change":"2026-02-22T20:50:35.237225+0000","last_active":"2026-02-22T20:50:35.237095+0000","last_peered":"2026-02-22T20:50:35.237095+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:50:35.232433+0000","last_became_peered":"2026-02-22T20:50:35.232433+0000","last_unstale":"2026-02-22T20:50:35.237095+0000","last_undegraded":"2026-02-22T20:50:35.237095+0000","last_fullsized":"2026-02-22T20:50:35.237095+0000","mapping_epoch":20,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10486,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130708+0000","last_change":"2026-02-22T20:50:34.368608+0000","last_active":"2026-02-22T20:50:35.130708+0000","last_peered":"2026-02-22T20:50:35.130708+0000","last_clean":"2026-02-22T20:50:31.300321+0000","last_became_active":"2026-02-22T20:50:34.362361+0000","last_became_peered":"2026-02-22T20:50:34.362361+0000","last_unstale":"2026-02-22T20:50:35.130708+0000","last_undegraded":"2026-02-22T20:50:35.130708+0000","last_fullsized":"2026-02-22T20:50:35.130708+0000","mapping_epoch":19,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":14595,"reported_epoch":21,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:50:55.020629+0000","last_change":"2026-02-22T20:50:34.426703+0000","last_active":"2026-02-22T20:50:55.020629+0000","last_peered":"2026-02-22T20:50:55.020629+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:50:34.404753+0000","last_became_peered":"2026-02-22T20:50:34.404753+0000","last_unstale":"2026-02-22T20:50:55.020629+0000","last_undegraded":"2026-02-22T20:50:55.020629+0000","last_fullsized":"2026-02-22T20:50:34.129679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17112,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1500,"num_bytes_recovered":1498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":1499},{"shards":"2,3","objects":8556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10206,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131737+0000","last_change":"2026-02-22T20:50:34.368334+0000","last_active":"2026-02-22T20:50:35.131737+0000","last_peered":"2026-02-22T20:50:35.131737+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:50:34.362168+0000","last_became_peered":"2026-02-22T20:50:34.362168+0000","last_unstale":"2026-02-22T20:50:35.131737+0000","last_undegraded":"2026-02-22T20:50:35.131737+0000","last_fullsized":"2026-02-22T20:50:35.131737+0000","mapping_epoch":19,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20336,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10293,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.130683+0000","last_change":"2026-02-22T20:50:34.366383+0000","last_active":"2026-02-22T20:50:35.130683+0000","last_peered":"2026-02-22T20:50:35.130683+0000","last_clean":"2026-02-22T20:50:30.477943+0000","last_became_active":"2026-02-22T20:50:34.356915+0000","last_became_peered":"2026-02-22T20:50:34.356915+0000","last_unstale":"2026-02-22T20:50:35.130683+0000","last_undegraded":"2026-02-22T20:50:35.130683+0000","last_fullsized":"2026-02-22T20:50:35.130683+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10271,"reported_epoch":21,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:50:35.131745+0000","last_change":"2026-02-22T20:50:34.375783+0000","last_active":"2026-02-22T20:50:35.131745+0000","last_peered":"2026-02-22T20:50:35.131745+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:50:34.369192+0000","last_became_peered":"2026-02-22T20:50:34.369192+0000","last_unstale":"2026-02-22T20:50:35.131745+0000","last_undegraded":"2026-02-22T20:50:35.131745+0000","last_fullsized":"2026-02-22T20:50:35.131745+0000","mapping_epoch":19,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":160444,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":682729472,"data_stored":747404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705884,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51300,"kb_used_data":6292,"kb_used_omap":11,"kb_used_meta":44980,"kb_avail":94320540,"statfs":{"total":96636764160,"available":96584232960,"internally_reserved":0,"allocated":6443008,"data_stored":60316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":46059690},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51164,"kb_used_data":6156,"kb_used_omap":11,"kb_used_meta":44980,"kb_avail":94320676,"statfs":{"total":96636764160,"available":96584372224,"internally_reserved":0,"allocated":6303744,"data_stored":60282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12118,"internal_metadata":46059690},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6270976,"data_stored":1531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6270976,"data_stored":1531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:50:57.153 INFO:tasks.osd_recovery:stopping 2,3 2026-02-22T20:50:57.153 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-22T20:50:57.154 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T20:50:59.020 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2026-02-22T20:51:03.255 INFO:tasks.ceph.osd.2:Stopped 2026-02-22T20:51:03.256 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-22T20:51:03.256 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T20:51:03.303 INFO:tasks.ceph.osd.3:Stopped 2026-02-22T20:51:03.303 INFO:tasks.osd_recovery:... 2026-02-22T20:51:03.303 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 3 2026-02-22T20:51:03.484 INFO:tasks.ceph.mon.a.trial084.stderr:2026-02-22T20:51:03.481+0000 7fb50627a6c0 -1 mon.a@0(leader).osd e21 definitely_dead 0 2026-02-22T20:51:04.240 INFO:teuthology.orchestra.run.trial084.stderr:marked down osd.2. marked down osd.3. 2026-02-22T20:51:04.259 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-22T20:51:04.259 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-22T20:51:04.366 INFO:teuthology.orchestra.run.trial084.stdout:38654705885 2026-02-22T20:51:04.367 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T20:51:04.368 INFO:teuthology.orchestra.run.trial084.stdout:38654705884 2026-02-22T20:51:04.369 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T20:51:04.589 INFO:teuthology.orchestra.run.trial084.stdout:38654705883 2026-02-22T20:51:04.590 INFO:teuthology.orchestra.run.trial084.stdout:38654705884 2026-02-22T20:51:04.603 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705884 got 38654705883 for osd.1 2026-02-22T20:51:04.603 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705885 got 38654705884 for osd.0 2026-02-22T20:51:04.625 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2026-02-22T20:51:05.603 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T20:51:05.604 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T20:51:05.785 INFO:teuthology.orchestra.run.trial084.stdout:38654705886 2026-02-22T20:51:05.786 INFO:teuthology.orchestra.run.trial084.stdout:38654705885 2026-02-22T20:51:05.798 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705885 got 38654705886 for osd.0 2026-02-22T20:51:05.799 DEBUG:teuthology.parallel:result is None 2026-02-22T20:51:05.800 INFO:tasks.osd_recovery.ceph_manager:need seq 38654705884 got 38654705885 for osd.1 2026-02-22T20:51:05.800 DEBUG:teuthology.parallel:result is None 2026-02-22T20:51:05.800 INFO:tasks.osd_recovery.ceph_manager:waiting for peering to complete or become blocked 2026-02-22T20:51:05.800 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:05.981 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:05.982 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:05.993 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":560,"stamp":"2026-02-22T20:51:05.240673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":40653,"num_objects":10088,"num_object_clones":0,"num_object_copies":20176,"num_objects_missing_on_primary":0,"num_objects_missing":18680,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":103920,"kb_used_data":13904,"kb_used_omap":24,"kb_used_meta":89959,"kb_avail":188639760,"statfs":{"total":193273528320,"available":193167114240,"internally_reserved":0,"allocated":14237696,"data_stored":127690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25538,"internal_metadata":92118078},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-331518,"num_objects":-71666,"num_object_clones":0,"num_object_copies":-143332,"num_objects_missing_on_primary":0,"num_objects_missing":18680,"num_objects_degraded":0,"num_objects_misplaced":-160444,"num_objects_unfound":0,"num_objects_dirty":-71666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-71700,"num_write_kb":-71938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1534,"num_bytes_recovered":-1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.132145"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":3,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.243020+0000","last_change":"2026-02-22T20:51:04.243020+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.243020+0000","last_undegraded":"2026-02-22T20:51:04.243020+0000","last_fullsized":"2026-02-22T20:51:04.243020+0000","mapping_epoch":22,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":2,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.242543+0000","last_change":"2026-02-22T20:51:04.242543+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.242543+0000","last_undegraded":"2026-02-22T20:51:04.242543+0000","last_fullsized":"2026-02-22T20:51:04.242543+0000","mapping_epoch":22,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":10284,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":3,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.241827+0000","last_change":"2026-02-22T20:51:04.241827+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.241827+0000","last_undegraded":"2026-02-22T20:51:04.241827+0000","last_fullsized":"2026-02-22T20:51:04.241827+0000","mapping_epoch":22,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":2,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.242984+0000","last_change":"2026-02-22T20:51:04.242984+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.242984+0000","last_undegraded":"2026-02-22T20:51:04.242984+0000","last_fullsized":"2026-02-22T20:51:04.242984+0000","mapping_epoch":22,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10092,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.242433+0000","last_change":"2026-02-22T20:51:04.242433+0000","last_active":"2026-02-22T20:50:33.132066+0000","last_peered":"2026-02-22T20:50:30.642683+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:51:04.242433+0000","last_undegraded":"2026-02-22T20:51:04.242433+0000","last_fullsized":"2026-02-22T20:51:04.242433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":8396,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":3,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.241852+0000","last_change":"2026-02-22T20:51:04.241852+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.241852+0000","last_undegraded":"2026-02-22T20:51:04.241852+0000","last_fullsized":"2026-02-22T20:51:04.241852+0000","mapping_epoch":22,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":2,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.243141+0000","last_change":"2026-02-22T20:51:04.243141+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.243141+0000","last_undegraded":"2026-02-22T20:51:04.243141+0000","last_fullsized":"2026-02-22T20:51:04.243141+0000","mapping_epoch":22,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":3,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.242255+0000","last_change":"2026-02-22T20:51:04.242255+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.242255+0000","last_undegraded":"2026-02-22T20:51:04.242255+0000","last_fullsized":"2026-02-22T20:51:04.242255+0000","mapping_epoch":22,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":40653,"num_objects":10088,"num_object_clones":0,"num_object_copies":20176,"num_objects_missing_on_primary":0,"num_objects_missing":18680,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":684048384,"data_stored":747726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705885,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705885,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51960,"kb_used_data":6952,"kb_used_omap":12,"kb_used_meta":44979,"kb_avail":94319880,"statfs":{"total":96636764160,"available":96583557120,"internally_reserved":0,"allocated":7118848,"data_stored":63845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12769,"internal_metadata":46059039},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705886,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51960,"kb_used_data":6952,"kb_used_omap":12,"kb_used_meta":44979,"kb_avail":94319880,"statfs":{"total":96636764160,"available":96583557120,"internally_reserved":0,"allocated":7118848,"data_stored":63845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12769,"internal_metadata":46059039},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6930432,"data_stored":1692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6930432,"data_stored":1692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:05.995 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:06.171 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:06.172 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:06.183 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":560,"stamp":"2026-02-22T20:51:05.240673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":40653,"num_objects":10088,"num_object_clones":0,"num_object_copies":20176,"num_objects_missing_on_primary":0,"num_objects_missing":18680,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":103920,"kb_used_data":13904,"kb_used_omap":24,"kb_used_meta":89959,"kb_avail":188639760,"statfs":{"total":193273528320,"available":193167114240,"internally_reserved":0,"allocated":14237696,"data_stored":127690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25538,"internal_metadata":92118078},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-331518,"num_objects":-71666,"num_object_clones":0,"num_object_copies":-143332,"num_objects_missing_on_primary":0,"num_objects_missing":18680,"num_objects_degraded":0,"num_objects_misplaced":-160444,"num_objects_unfound":0,"num_objects_dirty":-71666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-71700,"num_write_kb":-71938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1534,"num_bytes_recovered":-1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.132145"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":3,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.243020+0000","last_change":"2026-02-22T20:51:04.243020+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.243020+0000","last_undegraded":"2026-02-22T20:51:04.243020+0000","last_fullsized":"2026-02-22T20:51:04.243020+0000","mapping_epoch":22,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":2,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.242543+0000","last_change":"2026-02-22T20:51:04.242543+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.242543+0000","last_undegraded":"2026-02-22T20:51:04.242543+0000","last_fullsized":"2026-02-22T20:51:04.242543+0000","mapping_epoch":22,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":10284,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":3,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.241827+0000","last_change":"2026-02-22T20:51:04.241827+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.241827+0000","last_undegraded":"2026-02-22T20:51:04.241827+0000","last_fullsized":"2026-02-22T20:51:04.241827+0000","mapping_epoch":22,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":2,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.242984+0000","last_change":"2026-02-22T20:51:04.242984+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.242984+0000","last_undegraded":"2026-02-22T20:51:04.242984+0000","last_fullsized":"2026-02-22T20:51:04.242984+0000","mapping_epoch":22,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10092,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.242433+0000","last_change":"2026-02-22T20:51:04.242433+0000","last_active":"2026-02-22T20:50:33.132066+0000","last_peered":"2026-02-22T20:50:30.642683+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:51:04.242433+0000","last_undegraded":"2026-02-22T20:51:04.242433+0000","last_fullsized":"2026-02-22T20:51:04.242433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":8396,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":3,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.241852+0000","last_change":"2026-02-22T20:51:04.241852+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.241852+0000","last_undegraded":"2026-02-22T20:51:04.241852+0000","last_fullsized":"2026-02-22T20:51:04.241852+0000","mapping_epoch":22,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":2,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.243141+0000","last_change":"2026-02-22T20:51:04.243141+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.243141+0000","last_undegraded":"2026-02-22T20:51:04.243141+0000","last_fullsized":"2026-02-22T20:51:04.243141+0000","mapping_epoch":22,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":3,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.242255+0000","last_change":"2026-02-22T20:51:04.242255+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.242255+0000","last_undegraded":"2026-02-22T20:51:04.242255+0000","last_fullsized":"2026-02-22T20:51:04.242255+0000","mapping_epoch":22,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":40653,"num_objects":10088,"num_object_clones":0,"num_object_copies":20176,"num_objects_missing_on_primary":0,"num_objects_missing":18680,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":684048384,"data_stored":747726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705885,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705885,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51960,"kb_used_data":6952,"kb_used_omap":12,"kb_used_meta":44979,"kb_avail":94319880,"statfs":{"total":96636764160,"available":96583557120,"internally_reserved":0,"allocated":7118848,"data_stored":63845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12769,"internal_metadata":46059039},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705886,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51960,"kb_used_data":6952,"kb_used_omap":12,"kb_used_meta":44979,"kb_avail":94319880,"statfs":{"total":96636764160,"available":96583557120,"internally_reserved":0,"allocated":7118848,"data_stored":63845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12769,"internal_metadata":46059039},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6930432,"data_stored":1692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6930432,"data_stored":1692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:06.185 INFO:tasks.osd_recovery.ceph_manager:active or down! 2026-02-22T20:51:06.185 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:06.364 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:06.364 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:06.375 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":560,"stamp":"2026-02-22T20:51:05.240673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":40653,"num_objects":10088,"num_object_clones":0,"num_object_copies":20176,"num_objects_missing_on_primary":0,"num_objects_missing":18680,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":103920,"kb_used_data":13904,"kb_used_omap":24,"kb_used_meta":89959,"kb_avail":188639760,"statfs":{"total":193273528320,"available":193167114240,"internally_reserved":0,"allocated":14237696,"data_stored":127690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25538,"internal_metadata":92118078},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-331518,"num_objects":-71666,"num_object_clones":0,"num_object_copies":-143332,"num_objects_missing_on_primary":0,"num_objects_missing":18680,"num_objects_degraded":0,"num_objects_misplaced":-160444,"num_objects_unfound":0,"num_objects_dirty":-71666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-71700,"num_write_kb":-71938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1534,"num_bytes_recovered":-1551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.132145"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":3,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.243020+0000","last_change":"2026-02-22T20:51:04.243020+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.243020+0000","last_undegraded":"2026-02-22T20:51:04.243020+0000","last_fullsized":"2026-02-22T20:51:04.243020+0000","mapping_epoch":22,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":2,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.242543+0000","last_change":"2026-02-22T20:51:04.242543+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.242543+0000","last_undegraded":"2026-02-22T20:51:04.242543+0000","last_fullsized":"2026-02-22T20:51:04.242543+0000","mapping_epoch":22,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":10284,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":3,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.241827+0000","last_change":"2026-02-22T20:51:04.241827+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.241827+0000","last_undegraded":"2026-02-22T20:51:04.241827+0000","last_fullsized":"2026-02-22T20:51:04.241827+0000","mapping_epoch":22,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":2,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.242984+0000","last_change":"2026-02-22T20:51:04.242984+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.242984+0000","last_undegraded":"2026-02-22T20:51:04.242984+0000","last_fullsized":"2026-02-22T20:51:04.242984+0000","mapping_epoch":22,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10092,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.242433+0000","last_change":"2026-02-22T20:51:04.242433+0000","last_active":"2026-02-22T20:50:33.132066+0000","last_peered":"2026-02-22T20:50:30.642683+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:33:22.074095+0000","last_became_peered":"2026-02-22T20:33:22.074095+0000","last_unstale":"2026-02-22T20:51:04.242433+0000","last_undegraded":"2026-02-22T20:51:04.242433+0000","last_fullsized":"2026-02-22T20:51:04.242433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":8396,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":3,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.241852+0000","last_change":"2026-02-22T20:51:04.241852+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.241852+0000","last_undegraded":"2026-02-22T20:51:04.241852+0000","last_fullsized":"2026-02-22T20:51:04.241852+0000","mapping_epoch":22,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":2,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.243141+0000","last_change":"2026-02-22T20:51:04.243141+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.243141+0000","last_undegraded":"2026-02-22T20:51:04.243141+0000","last_fullsized":"2026-02-22T20:51:04.243141+0000","mapping_epoch":22,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":3,"reported_epoch":22,"state":"down","last_fresh":"2026-02-22T20:51:04.242255+0000","last_change":"2026-02-22T20:51:04.242255+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-22T20:51:04.242255+0000","last_undegraded":"2026-02-22T20:51:04.242255+0000","last_fullsized":"2026-02-22T20:51:04.242255+0000","mapping_epoch":22,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":40653,"num_objects":10088,"num_object_clones":0,"num_object_copies":20176,"num_objects_missing_on_primary":0,"num_objects_missing":18680,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":684048384,"data_stored":747726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":38654705885,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":38654705883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705885,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51960,"kb_used_data":6952,"kb_used_omap":12,"kb_used_meta":44979,"kb_avail":94319880,"statfs":{"total":96636764160,"available":96583557120,"internally_reserved":0,"allocated":7118848,"data_stored":63845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12769,"internal_metadata":46059039},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705886,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51960,"kb_used_data":6952,"kb_used_omap":12,"kb_used_meta":44979,"kb_avail":94319880,"statfs":{"total":96636764160,"available":96583557120,"internally_reserved":0,"allocated":7118848,"data_stored":63845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12769,"internal_metadata":46059039},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6930432,"data_stored":1692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6930432,"data_stored":1692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:06.377 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-22T20:51:06.377 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-22T20:51:06.379 INFO:tasks.ceph.osd.2:Started 2026-02-22T20:51:06.379 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2026-02-22T20:51:06.463 INFO:teuthology.orchestra.run.trial084.stderr:no valid command found; 10 closest matches: 2026-02-22T20:51:06.463 INFO:teuthology.orchestra.run.trial084.stderr:0 2026-02-22T20:51:06.463 INFO:teuthology.orchestra.run.trial084.stderr:1 2026-02-22T20:51:06.464 INFO:teuthology.orchestra.run.trial084.stderr:2 2026-02-22T20:51:06.464 INFO:teuthology.orchestra.run.trial084.stderr:abort 2026-02-22T20:51:06.464 INFO:teuthology.orchestra.run.trial084.stderr:assert 2026-02-22T20:51:06.464 INFO:teuthology.orchestra.run.trial084.stderr:bluestore collections 2026-02-22T20:51:06.464 INFO:teuthology.orchestra.run.trial084.stderr:bluestore compression stats [] 2026-02-22T20:51:06.464 INFO:teuthology.orchestra.run.trial084.stderr:bluestore list [] [] 2026-02-22T20:51:06.464 INFO:teuthology.orchestra.run.trial084.stderr:bluestore onode metadata 2026-02-22T20:51:06.464 INFO:teuthology.orchestra.run.trial084.stderr:config diff 2026-02-22T20:51:06.464 INFO:teuthology.orchestra.run.trial084.stderr:admin_socket: invalid command 2026-02-22T20:51:06.465 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-22T20:51:06.466 INFO:tasks.osd_recovery.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2026-02-22T20:51:06.592 INFO:tasks.ceph.osd.2.trial084.stderr:2026-02-22T20:51:06.589+0000 7f77e1aeb900 -1 Falling back to public interface 2026-02-22T20:51:11.467 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2026-02-22T20:51:11.550 INFO:teuthology.orchestra.run.trial084.stderr:no valid command found; 10 closest matches: 2026-02-22T20:51:11.550 INFO:teuthology.orchestra.run.trial084.stderr:0 2026-02-22T20:51:11.550 INFO:teuthology.orchestra.run.trial084.stderr:1 2026-02-22T20:51:11.550 INFO:teuthology.orchestra.run.trial084.stderr:2 2026-02-22T20:51:11.550 INFO:teuthology.orchestra.run.trial084.stderr:abort 2026-02-22T20:51:11.551 INFO:teuthology.orchestra.run.trial084.stderr:assert 2026-02-22T20:51:11.551 INFO:teuthology.orchestra.run.trial084.stderr:bluefs debug_inject_read_zeros 2026-02-22T20:51:11.551 INFO:teuthology.orchestra.run.trial084.stderr:bluefs files list 2026-02-22T20:51:11.551 INFO:teuthology.orchestra.run.trial084.stderr:bluefs stats 2026-02-22T20:51:11.551 INFO:teuthology.orchestra.run.trial084.stderr:bluestore allocator dump block 2026-02-22T20:51:11.551 INFO:teuthology.orchestra.run.trial084.stderr:bluestore allocator fragmentation block 2026-02-22T20:51:11.551 INFO:teuthology.orchestra.run.trial084.stderr:admin_socket: invalid command 2026-02-22T20:51:11.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-22T20:51:11.553 INFO:tasks.osd_recovery.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2026-02-22T20:51:11.663 INFO:tasks.ceph.osd.2.trial084.stderr:2026-02-22T20:51:11.660+0000 7f77e1aeb900 -1 osd.2 21 log_to_monitors true 2026-02-22T20:51:12.283 INFO:tasks.ceph.osd.2.trial084.stderr:2026-02-22T20:51:12.281+0000 7f77d90856c0 -1 osd.2 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T20:51:16.554 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2026-02-22T20:51:16.645 INFO:teuthology.orchestra.run.trial084.stdout:{ 2026-02-22T20:51:16.646 INFO:teuthology.orchestra.run.trial084.stdout: "ops": [], 2026-02-22T20:51:16.646 INFO:teuthology.orchestra.run.trial084.stdout: "num_ops": 0 2026-02-22T20:51:16.646 INFO:teuthology.orchestra.run.trial084.stdout:} 2026-02-22T20:51:16.656 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-22T20:51:16.656 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2026-02-22T20:51:16.658 INFO:tasks.ceph.osd.3:Started 2026-02-22T20:51:16.658 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2026-02-22T20:51:16.746 INFO:teuthology.orchestra.run.trial084.stderr:no valid command found; 10 closest matches: 2026-02-22T20:51:16.746 INFO:teuthology.orchestra.run.trial084.stderr:0 2026-02-22T20:51:16.746 INFO:teuthology.orchestra.run.trial084.stderr:1 2026-02-22T20:51:16.746 INFO:teuthology.orchestra.run.trial084.stderr:2 2026-02-22T20:51:16.746 INFO:teuthology.orchestra.run.trial084.stderr:abort 2026-02-22T20:51:16.746 INFO:teuthology.orchestra.run.trial084.stderr:assert 2026-02-22T20:51:16.746 INFO:teuthology.orchestra.run.trial084.stderr:bluestore collections 2026-02-22T20:51:16.747 INFO:teuthology.orchestra.run.trial084.stderr:bluestore compression stats [] 2026-02-22T20:51:16.747 INFO:teuthology.orchestra.run.trial084.stderr:bluestore list [] [] 2026-02-22T20:51:16.747 INFO:teuthology.orchestra.run.trial084.stderr:bluestore onode metadata 2026-02-22T20:51:16.747 INFO:teuthology.orchestra.run.trial084.stderr:config diff 2026-02-22T20:51:16.747 INFO:teuthology.orchestra.run.trial084.stderr:admin_socket: invalid command 2026-02-22T20:51:16.748 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-22T20:51:16.749 INFO:tasks.osd_recovery.ceph_manager:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2026-02-22T20:51:16.909 INFO:tasks.ceph.osd.3.trial084.stderr:2026-02-22T20:51:16.907+0000 7f43ec140900 -1 Falling back to public interface 2026-02-22T20:51:21.750 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2026-02-22T20:51:21.839 INFO:teuthology.orchestra.run.trial084.stderr:no valid command found; 10 closest matches: 2026-02-22T20:51:21.839 INFO:teuthology.orchestra.run.trial084.stderr:0 2026-02-22T20:51:21.839 INFO:teuthology.orchestra.run.trial084.stderr:1 2026-02-22T20:51:21.839 INFO:teuthology.orchestra.run.trial084.stderr:2 2026-02-22T20:51:21.839 INFO:teuthology.orchestra.run.trial084.stderr:abort 2026-02-22T20:51:21.839 INFO:teuthology.orchestra.run.trial084.stderr:assert 2026-02-22T20:51:21.839 INFO:teuthology.orchestra.run.trial084.stderr:bluefs debug_inject_read_zeros 2026-02-22T20:51:21.839 INFO:teuthology.orchestra.run.trial084.stderr:bluefs files list 2026-02-22T20:51:21.840 INFO:teuthology.orchestra.run.trial084.stderr:bluefs stats 2026-02-22T20:51:21.840 INFO:teuthology.orchestra.run.trial084.stderr:bluestore allocator dump block 2026-02-22T20:51:21.840 INFO:teuthology.orchestra.run.trial084.stderr:bluestore allocator fragmentation block 2026-02-22T20:51:21.840 INFO:teuthology.orchestra.run.trial084.stderr:admin_socket: invalid command 2026-02-22T20:51:21.841 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-22T20:51:21.842 INFO:tasks.osd_recovery.ceph_manager:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2026-02-22T20:51:22.297 INFO:tasks.ceph.osd.3.trial084.stderr:2026-02-22T20:51:22.295+0000 7f43ec140900 -1 osd.3 21 log_to_monitors true 2026-02-22T20:51:23.190 INFO:tasks.ceph.osd.3.trial084.stderr:2026-02-22T20:51:23.188+0000 7f43df6d26c0 -1 osd.3 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T20:51:26.842 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2026-02-22T20:51:26.929 INFO:teuthology.orchestra.run.trial084.stdout:{ 2026-02-22T20:51:26.929 INFO:teuthology.orchestra.run.trial084.stdout: "ops": [], 2026-02-22T20:51:26.929 INFO:teuthology.orchestra.run.trial084.stdout: "num_ops": 0 2026-02-22T20:51:26.929 INFO:teuthology.orchestra.run.trial084.stdout:} 2026-02-22T20:51:26.939 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2026-02-22T20:51:27.121 INFO:teuthology.orchestra.run.trial084.stdout:epoch 31 2026-02-22T20:51:27.121 INFO:teuthology.orchestra.run.trial084.stdout:fsid a4ea559c-fa00-404b-b21c-f34ee9bd0795 2026-02-22T20:51:27.121 INFO:teuthology.orchestra.run.trial084.stdout:created 2026-02-22T20:33:03.270442+0000 2026-02-22T20:51:27.121 INFO:teuthology.orchestra.run.trial084.stdout:modified 2026-02-22T20:51:26.172801+0000 2026-02-22T20:51:27.121 INFO:teuthology.orchestra.run.trial084.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2026-02-22T20:51:27.121 INFO:teuthology.orchestra.run.trial084.stdout:crush_version 6 2026-02-22T20:51:27.121 INFO:teuthology.orchestra.run.trial084.stdout:full_ratio 0.95 2026-02-22T20:51:27.121 INFO:teuthology.orchestra.run.trial084.stdout:backfillfull_ratio 0.9 2026-02-22T20:51:27.121 INFO:teuthology.orchestra.run.trial084.stdout:nearfull_ratio 0.85 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:require_min_compat_client luminous 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:min_compat_client jewel 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:require_osd_release tentacle 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:stretch_mode_enabled false 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 13 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.00 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:max_osd 4 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:osd.0 up in weight 1 up_from 9 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:10.20.193.84:6802/1867649424,v1:10.20.193.84:6803/1867649424] [v2:10.20.193.84:6804/1867649424,v1:10.20.193.84:6805/1867649424] exists,up ca475199-167f-4aba-960a-1e3782214a73 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:osd.1 up in weight 1 up_from 9 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:10.20.193.84:6811/2769530292,v1:10.20.193.84:6813/2769530292] [v2:10.20.193.84:6815/2769530292,v1:10.20.193.84:6818/2769530292] exists,up 32df476a-2b69-493e-a636-1ba8bf5268a2 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:osd.2 up out weight 0 up_from 25 up_thru 30 down_at 22 last_clean_interval [9,21) [v2:10.20.193.84:6810/524818824,v1:10.20.193.84:6812/524818824] [v2:10.20.193.84:6814/524818824,v1:10.20.193.84:6816/524818824] exists,up 196c3661-b3be-46a3-9f14-7908d9003447 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:osd.3 up out weight 0 up_from 29 up_thru 19 down_at 22 last_clean_interval [9,21) [v2:10.20.193.84:6826/1452505971,v1:10.20.193.84:6827/1452505971] [v2:10.20.193.84:6828/1452505971,v1:10.20.193.84:6829/1452505971] exists,up bdc6271f-361f-4362-b050-080c613ab56c 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:pg_temp 1.0 [2,3] 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:pg_temp 1.1 [2,3] 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:pg_temp 1.2 [2,3] 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:pg_temp 1.3 [2] 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:pg_temp 1.4 [2,3] 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:pg_temp 1.5 [2,3] 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:pg_temp 1.6 [2,3] 2026-02-22T20:51:27.122 INFO:teuthology.orchestra.run.trial084.stdout:pg_temp 1.7 [2,3] 2026-02-22T20:51:27.135 INFO:tasks.osd_recovery.ceph_manager:['osd.0 up in weight 1 up_from 9 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:10.20.193.84:6802/1867649424,v1:10.20.193.84:6803/1867649424] [v2:10.20.193.84:6804/1867649424,v1:10.20.193.84:6805/1867649424] exists,up ca475199-167f-4aba-960a-1e3782214a73', 'osd.1 up in weight 1 up_from 9 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:10.20.193.84:6811/2769530292,v1:10.20.193.84:6813/2769530292] [v2:10.20.193.84:6815/2769530292,v1:10.20.193.84:6818/2769530292] exists,up 32df476a-2b69-493e-a636-1ba8bf5268a2', 'osd.2 up out weight 0 up_from 25 up_thru 30 down_at 22 last_clean_interval [9,21) [v2:10.20.193.84:6810/524818824,v1:10.20.193.84:6812/524818824] [v2:10.20.193.84:6814/524818824,v1:10.20.193.84:6816/524818824] exists,up 196c3661-b3be-46a3-9f14-7908d9003447', 'osd.3 up out weight 0 up_from 29 up_thru 19 down_at 22 last_clean_interval [9,21) [v2:10.20.193.84:6826/1452505971,v1:10.20.193.84:6827/1452505971] [v2:10.20.193.84:6828/1452505971,v1:10.20.193.84:6829/1452505971] exists,up bdc6271f-361f-4362-b050-080c613ab56c'] 2026-02-22T20:51:27.135 INFO:tasks.osd_recovery:all are up! 2026-02-22T20:51:27.135 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 0 2026-02-22T20:51:27.233 INFO:teuthology.orchestra.run.trial084.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T20:51:27.244 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 debug kick_recovery_wq 0 2026-02-22T20:51:27.341 INFO:teuthology.orchestra.run.trial084.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T20:51:27.353 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 0 2026-02-22T20:51:27.448 INFO:teuthology.orchestra.run.trial084.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T20:51:27.459 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 debug kick_recovery_wq 0 2026-02-22T20:51:27.556 INFO:teuthology.orchestra.run.trial084.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T20:51:27.567 INFO:tasks.osd_recovery.ceph_manager:waiting for clean 2026-02-22T20:51:27.567 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:27.746 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:27.746 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:27.759 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":579,"stamp":"2026-02-22T20:51:27.112384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79354,"num_objects_misplaced":79354,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":711,"num_bytes_recovered":728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":8,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":111376,"kb_used_data":21360,"kb_used_omap":33,"kb_used_meta":89950,"kb_avail":188632304,"statfs":{"total":193273528320,"available":193159479296,"internally_reserved":0,"allocated":21872640,"data_stored":176652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":34001,"internal_metadata":92109615},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000559"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10239,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.275793+0000","last_change":"2026-02-22T20:51:15.275869+0000","last_active":"2026-02-22T20:51:15.275793+0000","last_peered":"2026-02-22T20:51:15.275793+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:15.270783+0000","last_became_peered":"2026-02-22T20:51:15.270783+0000","last_unstale":"2026-02-22T20:51:15.275793+0000","last_undegraded":"2026-02-22T20:51:15.268898+0000","last_fullsized":"2026-02-22T20:51:15.268704+0000","mapping_epoch":26,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10205,"num_objects_misplaced":10205,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10365,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.277110+0000","last_change":"2026-02-22T20:51:15.277110+0000","last_active":"2026-02-22T20:51:15.277110+0000","last_peered":"2026-02-22T20:51:15.277110+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:15.270821+0000","last_became_peered":"2026-02-22T20:51:15.270821+0000","last_unstale":"2026-02-22T20:51:15.277110+0000","last_undegraded":"2026-02-22T20:51:15.268762+0000","last_fullsized":"2026-02-22T20:51:15.268665+0000","mapping_epoch":26,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10283,"num_objects_misplaced":10283,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10156,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.275487+0000","last_change":"2026-02-22T20:51:15.275611+0000","last_active":"2026-02-22T20:51:15.275487+0000","last_peered":"2026-02-22T20:51:15.275487+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:15.270730+0000","last_became_peered":"2026-02-22T20:51:15.270730+0000","last_unstale":"2026-02-22T20:51:15.275487+0000","last_undegraded":"2026-02-22T20:51:15.268858+0000","last_fullsized":"2026-02-22T20:51:15.268646+0000","mapping_epoch":26,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10101,"num_objects_misplaced":10101,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10482,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.275631+0000","last_change":"2026-02-22T20:51:15.275631+0000","last_active":"2026-02-22T20:51:15.275631+0000","last_peered":"2026-02-22T20:51:15.275631+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:15.270568+0000","last_became_peered":"2026-02-22T20:51:15.270568+0000","last_unstale":"2026-02-22T20:51:15.275631+0000","last_undegraded":"2026-02-22T20:51:15.268800+0000","last_fullsized":"2026-02-22T20:51:15.268627+0000","mapping_epoch":26,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10432,"num_objects_misplaced":10432,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":12062,"reported_epoch":27,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2026-02-22T20:51:22.250339+0000","last_change":"2026-02-22T20:51:15.340697+0000","last_active":"2026-02-22T20:51:22.250339+0000","last_peered":"2026-02-22T20:51:22.250339+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:22.250339+0000","last_undegraded":"2026-02-22T20:51:15.330659+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7744,"num_objects_misplaced":7744,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":653,"num_bytes_recovered":652,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2311}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10386,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.340375+0000","last_change":"2026-02-22T20:51:15.339890+0000","last_active":"2026-02-22T20:51:15.340375+0000","last_peered":"2026-02-22T20:51:15.340375+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:15.270352+0000","last_became_peered":"2026-02-22T20:51:15.270352+0000","last_unstale":"2026-02-22T20:51:15.340375+0000","last_undegraded":"2026-02-22T20:51:15.268770+0000","last_fullsized":"2026-02-22T20:51:15.268617+0000","mapping_epoch":26,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10112,"num_objects_misplaced":10112,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10288,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.276073+0000","last_change":"2026-02-22T20:51:15.276073+0000","last_active":"2026-02-22T20:51:15.276073+0000","last_peered":"2026-02-22T20:51:15.276073+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:15.270322+0000","last_became_peered":"2026-02-22T20:51:15.270322+0000","last_unstale":"2026-02-22T20:51:15.276073+0000","last_undegraded":"2026-02-22T20:51:15.268722+0000","last_fullsized":"2026-02-22T20:51:15.268618+0000","mapping_epoch":26,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10238,"num_objects_misplaced":10238,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10278,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.277023+0000","last_change":"2026-02-22T20:51:15.277172+0000","last_active":"2026-02-22T20:51:15.277023+0000","last_peered":"2026-02-22T20:51:15.277023+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:15.270776+0000","last_became_peered":"2026-02-22T20:51:15.270776+0000","last_unstale":"2026-02-22T20:51:15.277023+0000","last_undegraded":"2026-02-22T20:51:15.268843+0000","last_fullsized":"2026-02-22T20:51:15.268604+0000","mapping_epoch":26,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10239,"num_objects_misplaced":10239,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79354,"num_objects_misplaced":79354,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":711,"num_bytes_recovered":728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":692174848,"data_stored":749710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":8,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":1,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182403,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55592,"kb_used_data":10584,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94316248,"statfs":{"total":96636764160,"available":96579837952,"internally_reserved":0,"allocated":10838016,"data_stored":86423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16675,"internal_metadata":46055133},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55784,"kb_used_data":10776,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94316056,"statfs":{"total":96636764160,"available":96579641344,"internally_reserved":0,"allocated":11034624,"data_stored":90229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10993664,"data_stored":2684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10993664,"data_stored":2684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:27.759 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:27.942 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:27.942 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:27.956 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":579,"stamp":"2026-02-22T20:51:27.112384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79354,"num_objects_misplaced":79354,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":711,"num_bytes_recovered":728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":8,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":111376,"kb_used_data":21360,"kb_used_omap":33,"kb_used_meta":89950,"kb_avail":188632304,"statfs":{"total":193273528320,"available":193159479296,"internally_reserved":0,"allocated":21872640,"data_stored":176652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":34001,"internal_metadata":92109615},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000559"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10239,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.275793+0000","last_change":"2026-02-22T20:51:15.275869+0000","last_active":"2026-02-22T20:51:15.275793+0000","last_peered":"2026-02-22T20:51:15.275793+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:15.270783+0000","last_became_peered":"2026-02-22T20:51:15.270783+0000","last_unstale":"2026-02-22T20:51:15.275793+0000","last_undegraded":"2026-02-22T20:51:15.268898+0000","last_fullsized":"2026-02-22T20:51:15.268704+0000","mapping_epoch":26,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10205,"num_objects_misplaced":10205,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10365,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.277110+0000","last_change":"2026-02-22T20:51:15.277110+0000","last_active":"2026-02-22T20:51:15.277110+0000","last_peered":"2026-02-22T20:51:15.277110+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:15.270821+0000","last_became_peered":"2026-02-22T20:51:15.270821+0000","last_unstale":"2026-02-22T20:51:15.277110+0000","last_undegraded":"2026-02-22T20:51:15.268762+0000","last_fullsized":"2026-02-22T20:51:15.268665+0000","mapping_epoch":26,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10283,"num_objects_misplaced":10283,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10156,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.275487+0000","last_change":"2026-02-22T20:51:15.275611+0000","last_active":"2026-02-22T20:51:15.275487+0000","last_peered":"2026-02-22T20:51:15.275487+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:15.270730+0000","last_became_peered":"2026-02-22T20:51:15.270730+0000","last_unstale":"2026-02-22T20:51:15.275487+0000","last_undegraded":"2026-02-22T20:51:15.268858+0000","last_fullsized":"2026-02-22T20:51:15.268646+0000","mapping_epoch":26,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10101,"num_objects_misplaced":10101,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10482,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.275631+0000","last_change":"2026-02-22T20:51:15.275631+0000","last_active":"2026-02-22T20:51:15.275631+0000","last_peered":"2026-02-22T20:51:15.275631+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:15.270568+0000","last_became_peered":"2026-02-22T20:51:15.270568+0000","last_unstale":"2026-02-22T20:51:15.275631+0000","last_undegraded":"2026-02-22T20:51:15.268800+0000","last_fullsized":"2026-02-22T20:51:15.268627+0000","mapping_epoch":26,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10432,"num_objects_misplaced":10432,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":12062,"reported_epoch":27,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2026-02-22T20:51:22.250339+0000","last_change":"2026-02-22T20:51:15.340697+0000","last_active":"2026-02-22T20:51:22.250339+0000","last_peered":"2026-02-22T20:51:22.250339+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:22.250339+0000","last_undegraded":"2026-02-22T20:51:15.330659+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7744,"num_objects_misplaced":7744,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":653,"num_bytes_recovered":652,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2311}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10386,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.340375+0000","last_change":"2026-02-22T20:51:15.339890+0000","last_active":"2026-02-22T20:51:15.340375+0000","last_peered":"2026-02-22T20:51:15.340375+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:15.270352+0000","last_became_peered":"2026-02-22T20:51:15.270352+0000","last_unstale":"2026-02-22T20:51:15.340375+0000","last_undegraded":"2026-02-22T20:51:15.268770+0000","last_fullsized":"2026-02-22T20:51:15.268617+0000","mapping_epoch":26,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10112,"num_objects_misplaced":10112,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10288,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.276073+0000","last_change":"2026-02-22T20:51:15.276073+0000","last_active":"2026-02-22T20:51:15.276073+0000","last_peered":"2026-02-22T20:51:15.276073+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:15.270322+0000","last_became_peered":"2026-02-22T20:51:15.270322+0000","last_unstale":"2026-02-22T20:51:15.276073+0000","last_undegraded":"2026-02-22T20:51:15.268722+0000","last_fullsized":"2026-02-22T20:51:15.268618+0000","mapping_epoch":26,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10238,"num_objects_misplaced":10238,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10278,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.277023+0000","last_change":"2026-02-22T20:51:15.277172+0000","last_active":"2026-02-22T20:51:15.277023+0000","last_peered":"2026-02-22T20:51:15.277023+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:15.270776+0000","last_became_peered":"2026-02-22T20:51:15.270776+0000","last_unstale":"2026-02-22T20:51:15.277023+0000","last_undegraded":"2026-02-22T20:51:15.268843+0000","last_fullsized":"2026-02-22T20:51:15.268604+0000","mapping_epoch":26,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10239,"num_objects_misplaced":10239,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79354,"num_objects_misplaced":79354,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":711,"num_bytes_recovered":728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":692174848,"data_stored":749710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":8,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":1,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182403,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55592,"kb_used_data":10584,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94316248,"statfs":{"total":96636764160,"available":96579837952,"internally_reserved":0,"allocated":10838016,"data_stored":86423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16675,"internal_metadata":46055133},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55784,"kb_used_data":10776,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94316056,"statfs":{"total":96636764160,"available":96579641344,"internally_reserved":0,"allocated":11034624,"data_stored":90229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10993664,"data_stored":2684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10993664,"data_stored":2684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:27.957 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:28.141 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:28.142 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:28.157 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":579,"stamp":"2026-02-22T20:51:27.112384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79354,"num_objects_misplaced":79354,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":711,"num_bytes_recovered":728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":8,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":111376,"kb_used_data":21360,"kb_used_omap":33,"kb_used_meta":89950,"kb_avail":188632304,"statfs":{"total":193273528320,"available":193159479296,"internally_reserved":0,"allocated":21872640,"data_stored":176652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":34001,"internal_metadata":92109615},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000559"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10239,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.275793+0000","last_change":"2026-02-22T20:51:15.275869+0000","last_active":"2026-02-22T20:51:15.275793+0000","last_peered":"2026-02-22T20:51:15.275793+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:15.270783+0000","last_became_peered":"2026-02-22T20:51:15.270783+0000","last_unstale":"2026-02-22T20:51:15.275793+0000","last_undegraded":"2026-02-22T20:51:15.268898+0000","last_fullsized":"2026-02-22T20:51:15.268704+0000","mapping_epoch":26,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10205,"num_objects_misplaced":10205,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10365,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.277110+0000","last_change":"2026-02-22T20:51:15.277110+0000","last_active":"2026-02-22T20:51:15.277110+0000","last_peered":"2026-02-22T20:51:15.277110+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:15.270821+0000","last_became_peered":"2026-02-22T20:51:15.270821+0000","last_unstale":"2026-02-22T20:51:15.277110+0000","last_undegraded":"2026-02-22T20:51:15.268762+0000","last_fullsized":"2026-02-22T20:51:15.268665+0000","mapping_epoch":26,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10283,"num_objects_misplaced":10283,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10156,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.275487+0000","last_change":"2026-02-22T20:51:15.275611+0000","last_active":"2026-02-22T20:51:15.275487+0000","last_peered":"2026-02-22T20:51:15.275487+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:15.270730+0000","last_became_peered":"2026-02-22T20:51:15.270730+0000","last_unstale":"2026-02-22T20:51:15.275487+0000","last_undegraded":"2026-02-22T20:51:15.268858+0000","last_fullsized":"2026-02-22T20:51:15.268646+0000","mapping_epoch":26,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10101,"num_objects_misplaced":10101,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10482,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.275631+0000","last_change":"2026-02-22T20:51:15.275631+0000","last_active":"2026-02-22T20:51:15.275631+0000","last_peered":"2026-02-22T20:51:15.275631+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:15.270568+0000","last_became_peered":"2026-02-22T20:51:15.270568+0000","last_unstale":"2026-02-22T20:51:15.275631+0000","last_undegraded":"2026-02-22T20:51:15.268800+0000","last_fullsized":"2026-02-22T20:51:15.268627+0000","mapping_epoch":26,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10432,"num_objects_misplaced":10432,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":12062,"reported_epoch":27,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2026-02-22T20:51:22.250339+0000","last_change":"2026-02-22T20:51:15.340697+0000","last_active":"2026-02-22T20:51:22.250339+0000","last_peered":"2026-02-22T20:51:22.250339+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:22.250339+0000","last_undegraded":"2026-02-22T20:51:15.330659+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7744,"num_objects_misplaced":7744,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":653,"num_bytes_recovered":652,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2311}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10386,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.340375+0000","last_change":"2026-02-22T20:51:15.339890+0000","last_active":"2026-02-22T20:51:15.340375+0000","last_peered":"2026-02-22T20:51:15.340375+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:15.270352+0000","last_became_peered":"2026-02-22T20:51:15.270352+0000","last_unstale":"2026-02-22T20:51:15.340375+0000","last_undegraded":"2026-02-22T20:51:15.268770+0000","last_fullsized":"2026-02-22T20:51:15.268617+0000","mapping_epoch":26,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10112,"num_objects_misplaced":10112,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10288,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.276073+0000","last_change":"2026-02-22T20:51:15.276073+0000","last_active":"2026-02-22T20:51:15.276073+0000","last_peered":"2026-02-22T20:51:15.276073+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:15.270322+0000","last_became_peered":"2026-02-22T20:51:15.270322+0000","last_unstale":"2026-02-22T20:51:15.276073+0000","last_undegraded":"2026-02-22T20:51:15.268722+0000","last_fullsized":"2026-02-22T20:51:15.268618+0000","mapping_epoch":26,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10238,"num_objects_misplaced":10238,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10278,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.277023+0000","last_change":"2026-02-22T20:51:15.277172+0000","last_active":"2026-02-22T20:51:15.277023+0000","last_peered":"2026-02-22T20:51:15.277023+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:15.270776+0000","last_became_peered":"2026-02-22T20:51:15.270776+0000","last_unstale":"2026-02-22T20:51:15.277023+0000","last_undegraded":"2026-02-22T20:51:15.268843+0000","last_fullsized":"2026-02-22T20:51:15.268604+0000","mapping_epoch":26,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10239,"num_objects_misplaced":10239,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79354,"num_objects_misplaced":79354,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":711,"num_bytes_recovered":728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":692174848,"data_stored":749710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":8,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":1,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182403,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55592,"kb_used_data":10584,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94316248,"statfs":{"total":96636764160,"available":96579837952,"internally_reserved":0,"allocated":10838016,"data_stored":86423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16675,"internal_metadata":46055133},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55784,"kb_used_data":10776,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94316056,"statfs":{"total":96636764160,"available":96579641344,"internally_reserved":0,"allocated":11034624,"data_stored":90229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10993664,"data_stored":2684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10993664,"data_stored":2684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:28.158 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:51:28.158 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10239, 'reported_epoch': 27, 'state': 'active+undersized+degraded+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:15.275793+0000', 'last_change': '2026-02-22T20:51:15.275869+0000', 'last_active': '2026-02-22T20:51:15.275793+0000', 'last_peered': '2026-02-22T20:51:15.275793+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:15.270783+0000', 'last_became_peered': '2026-02-22T20:51:15.270783+0000', 'last_unstale': '2026-02-22T20:51:15.275793+0000', 'last_undegraded': '2026-02-22T20:51:15.268898+0000', 'last_fullsized': '2026-02-22T20:51:15.268704+0000', 'mapping_epoch': 26, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 10205, 'num_objects_misplaced': 10205, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2'], 'object_location_counts': [{'shards': '2', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:28.158 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:51:28.158 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10365, 'reported_epoch': 27, 'state': 'active+undersized+degraded+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:15.277110+0000', 'last_change': '2026-02-22T20:51:15.277110+0000', 'last_active': '2026-02-22T20:51:15.277110+0000', 'last_peered': '2026-02-22T20:51:15.277110+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:15.270821+0000', 'last_became_peered': '2026-02-22T20:51:15.270821+0000', 'last_unstale': '2026-02-22T20:51:15.277110+0000', 'last_undegraded': '2026-02-22T20:51:15.268762+0000', 'last_fullsized': '2026-02-22T20:51:15.268665+0000', 'mapping_epoch': 26, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 10283, 'num_objects_misplaced': 10283, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2'], 'object_location_counts': [{'shards': '2', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:28.158 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:51:28.158 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10156, 'reported_epoch': 27, 'state': 'active+undersized+degraded+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:15.275487+0000', 'last_change': '2026-02-22T20:51:15.275611+0000', 'last_active': '2026-02-22T20:51:15.275487+0000', 'last_peered': '2026-02-22T20:51:15.275487+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:15.270730+0000', 'last_became_peered': '2026-02-22T20:51:15.270730+0000', 'last_unstale': '2026-02-22T20:51:15.275487+0000', 'last_undegraded': '2026-02-22T20:51:15.268858+0000', 'last_fullsized': '2026-02-22T20:51:15.268646+0000', 'mapping_epoch': 26, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 10101, 'num_objects_misplaced': 10101, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2'], 'object_location_counts': [{'shards': '2', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:28.158 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:51:28.158 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10482, 'reported_epoch': 27, 'state': 'active+undersized+degraded+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:15.275631+0000', 'last_change': '2026-02-22T20:51:15.275631+0000', 'last_active': '2026-02-22T20:51:15.275631+0000', 'last_peered': '2026-02-22T20:51:15.275631+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:15.270568+0000', 'last_became_peered': '2026-02-22T20:51:15.270568+0000', 'last_unstale': '2026-02-22T20:51:15.275631+0000', 'last_undegraded': '2026-02-22T20:51:15.268800+0000', 'last_fullsized': '2026-02-22T20:51:15.268627+0000', 'mapping_epoch': 26, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 10432, 'num_objects_misplaced': 10432, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2'], 'object_location_counts': [{'shards': '2', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:28.159 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:51:28.159 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 12062, 'reported_epoch': 27, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2026-02-22T20:51:22.250339+0000', 'last_change': '2026-02-22T20:51:15.340697+0000', 'last_active': '2026-02-22T20:51:22.250339+0000', 'last_peered': '2026-02-22T20:51:22.250339+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:51:15.334499+0000', 'last_became_peered': '2026-02-22T20:51:15.334499+0000', 'last_unstale': '2026-02-22T20:51:22.250339+0000', 'last_undegraded': '2026-02-22T20:51:15.330659+0000', 'last_fullsized': '2026-02-22T20:51:15.268613+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7744, 'num_objects_misplaced': 7744, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 653, 'num_bytes_recovered': 652, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2'], 'object_location_counts': [{'shards': '2', 'objects': 2311}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:28.159 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:51:28.159 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10386, 'reported_epoch': 27, 'state': 'active+undersized+degraded+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:15.340375+0000', 'last_change': '2026-02-22T20:51:15.339890+0000', 'last_active': '2026-02-22T20:51:15.340375+0000', 'last_peered': '2026-02-22T20:51:15.340375+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:15.270352+0000', 'last_became_peered': '2026-02-22T20:51:15.270352+0000', 'last_unstale': '2026-02-22T20:51:15.340375+0000', 'last_undegraded': '2026-02-22T20:51:15.268770+0000', 'last_fullsized': '2026-02-22T20:51:15.268617+0000', 'mapping_epoch': 26, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 10112, 'num_objects_misplaced': 10112, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2'], 'object_location_counts': [{'shards': '2', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:28.159 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:51:28.159 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10288, 'reported_epoch': 27, 'state': 'active+undersized+degraded+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:15.276073+0000', 'last_change': '2026-02-22T20:51:15.276073+0000', 'last_active': '2026-02-22T20:51:15.276073+0000', 'last_peered': '2026-02-22T20:51:15.276073+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:15.270322+0000', 'last_became_peered': '2026-02-22T20:51:15.270322+0000', 'last_unstale': '2026-02-22T20:51:15.276073+0000', 'last_undegraded': '2026-02-22T20:51:15.268722+0000', 'last_fullsized': '2026-02-22T20:51:15.268618+0000', 'mapping_epoch': 26, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 10238, 'num_objects_misplaced': 10238, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2], 'avail_no_missing': ['2'], 'object_location_counts': [{'shards': '2', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:28.159 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:51:28.159 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10278, 'reported_epoch': 27, 'state': 'active+undersized+degraded+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:15.277023+0000', 'last_change': '2026-02-22T20:51:15.277172+0000', 'last_active': '2026-02-22T20:51:15.277023+0000', 'last_peered': '2026-02-22T20:51:15.277023+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:15.270776+0000', 'last_became_peered': '2026-02-22T20:51:15.270776+0000', 'last_unstale': '2026-02-22T20:51:15.277023+0000', 'last_undegraded': '2026-02-22T20:51:15.268843+0000', 'last_fullsized': '2026-02-22T20:51:15.268604+0000', 'mapping_epoch': 26, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 10239, 'num_objects_misplaced': 10239, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2'], 'object_location_counts': [{'shards': '2', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:28.159 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:51:28.383 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:28.383 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 79354/163508 objects degraded (48.532%), 8 pgs degraded","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1105,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":31,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+undersized+degraded+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":114049024,"bytes_avail":193159479296,"bytes_total":193273528320,"degraded_objects":79354,"degraded_total":163508,"degraded_ratio":0.48532181911588423,"misplaced_objects":79354,"misplaced_total":163508,"misplaced_ratio":0.48532181911588423},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:51:28.396 INFO:tasks.osd_recovery.ceph_manager:no progress seen, keeping timeout for now 2026-02-22T20:51:28.397 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:28.577 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:28.578 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:28.591 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":579,"stamp":"2026-02-22T20:51:27.112384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79354,"num_objects_misplaced":79354,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":711,"num_bytes_recovered":728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":8,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":111376,"kb_used_data":21360,"kb_used_omap":33,"kb_used_meta":89950,"kb_avail":188632304,"statfs":{"total":193273528320,"available":193159479296,"internally_reserved":0,"allocated":21872640,"data_stored":176652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":34001,"internal_metadata":92109615},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000559"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10239,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.275793+0000","last_change":"2026-02-22T20:51:15.275869+0000","last_active":"2026-02-22T20:51:15.275793+0000","last_peered":"2026-02-22T20:51:15.275793+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:15.270783+0000","last_became_peered":"2026-02-22T20:51:15.270783+0000","last_unstale":"2026-02-22T20:51:15.275793+0000","last_undegraded":"2026-02-22T20:51:15.268898+0000","last_fullsized":"2026-02-22T20:51:15.268704+0000","mapping_epoch":26,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10205,"num_objects_misplaced":10205,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10365,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.277110+0000","last_change":"2026-02-22T20:51:15.277110+0000","last_active":"2026-02-22T20:51:15.277110+0000","last_peered":"2026-02-22T20:51:15.277110+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:15.270821+0000","last_became_peered":"2026-02-22T20:51:15.270821+0000","last_unstale":"2026-02-22T20:51:15.277110+0000","last_undegraded":"2026-02-22T20:51:15.268762+0000","last_fullsized":"2026-02-22T20:51:15.268665+0000","mapping_epoch":26,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10283,"num_objects_misplaced":10283,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10156,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.275487+0000","last_change":"2026-02-22T20:51:15.275611+0000","last_active":"2026-02-22T20:51:15.275487+0000","last_peered":"2026-02-22T20:51:15.275487+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:15.270730+0000","last_became_peered":"2026-02-22T20:51:15.270730+0000","last_unstale":"2026-02-22T20:51:15.275487+0000","last_undegraded":"2026-02-22T20:51:15.268858+0000","last_fullsized":"2026-02-22T20:51:15.268646+0000","mapping_epoch":26,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10101,"num_objects_misplaced":10101,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10482,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.275631+0000","last_change":"2026-02-22T20:51:15.275631+0000","last_active":"2026-02-22T20:51:15.275631+0000","last_peered":"2026-02-22T20:51:15.275631+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:15.270568+0000","last_became_peered":"2026-02-22T20:51:15.270568+0000","last_unstale":"2026-02-22T20:51:15.275631+0000","last_undegraded":"2026-02-22T20:51:15.268800+0000","last_fullsized":"2026-02-22T20:51:15.268627+0000","mapping_epoch":26,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10432,"num_objects_misplaced":10432,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":12062,"reported_epoch":27,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2026-02-22T20:51:22.250339+0000","last_change":"2026-02-22T20:51:15.340697+0000","last_active":"2026-02-22T20:51:22.250339+0000","last_peered":"2026-02-22T20:51:22.250339+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:22.250339+0000","last_undegraded":"2026-02-22T20:51:15.330659+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7744,"num_objects_misplaced":7744,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":653,"num_bytes_recovered":652,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2311}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10386,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.340375+0000","last_change":"2026-02-22T20:51:15.339890+0000","last_active":"2026-02-22T20:51:15.340375+0000","last_peered":"2026-02-22T20:51:15.340375+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:15.270352+0000","last_became_peered":"2026-02-22T20:51:15.270352+0000","last_unstale":"2026-02-22T20:51:15.340375+0000","last_undegraded":"2026-02-22T20:51:15.268770+0000","last_fullsized":"2026-02-22T20:51:15.268617+0000","mapping_epoch":26,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10112,"num_objects_misplaced":10112,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10288,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.276073+0000","last_change":"2026-02-22T20:51:15.276073+0000","last_active":"2026-02-22T20:51:15.276073+0000","last_peered":"2026-02-22T20:51:15.276073+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:15.270322+0000","last_became_peered":"2026-02-22T20:51:15.270322+0000","last_unstale":"2026-02-22T20:51:15.276073+0000","last_undegraded":"2026-02-22T20:51:15.268722+0000","last_fullsized":"2026-02-22T20:51:15.268618+0000","mapping_epoch":26,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10238,"num_objects_misplaced":10238,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10278,"reported_epoch":27,"state":"active+undersized+degraded+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:15.277023+0000","last_change":"2026-02-22T20:51:15.277172+0000","last_active":"2026-02-22T20:51:15.277023+0000","last_peered":"2026-02-22T20:51:15.277023+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:15.270776+0000","last_became_peered":"2026-02-22T20:51:15.270776+0000","last_unstale":"2026-02-22T20:51:15.277023+0000","last_undegraded":"2026-02-22T20:51:15.268843+0000","last_fullsized":"2026-02-22T20:51:15.268604+0000","mapping_epoch":26,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10239,"num_objects_misplaced":10239,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79354,"num_objects_misplaced":79354,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":711,"num_bytes_recovered":728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":692174848,"data_stored":749710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":8,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":1,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182403,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55592,"kb_used_data":10584,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94316248,"statfs":{"total":96636764160,"available":96579837952,"internally_reserved":0,"allocated":10838016,"data_stored":86423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16675,"internal_metadata":46055133},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55784,"kb_used_data":10776,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94316056,"statfs":{"total":96636764160,"available":96579641344,"internally_reserved":0,"allocated":11034624,"data_stored":90229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10993664,"data_stored":2684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10993664,"data_stored":2684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:31.593 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:31.775 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:31.776 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:31.789 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":581,"stamp":"2026-02-22T20:51:31.112915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":157672,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1229,"num_bytes_recovered":1246,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":115536,"kb_used_data":25520,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188628144,"statfs":{"total":193273528320,"available":193155219456,"internally_reserved":0,"allocated":26132480,"data_stored":195520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-79354,"num_objects_misplaced":78318,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.945117"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":13624,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:27.259826+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:27.259826+0000","last_peered":"2026-02-22T20:51:27.259826+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:27.259826+0000","last_undegraded":"2026-02-22T20:51:27.259826+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14452,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1171,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":2829},{"shards":"2,3","objects":7226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":157672,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1229,"num_bytes_recovered":1246,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696442880,"data_stored":750752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051586,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182404,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57660,"kb_used_data":12652,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94314180,"statfs":{"total":96636764160,"available":96577720320,"internally_reserved":0,"allocated":12955648,"data_stored":97733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57876,"kb_used_data":12868,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94313964,"statfs":{"total":96636764160,"available":96577499136,"internally_reserved":0,"allocated":13176832,"data_stored":97787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13127680,"data_stored":3205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13127680,"data_stored":3205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:31.790 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:31.975 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:31.975 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:31.988 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":581,"stamp":"2026-02-22T20:51:31.112915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":157672,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1229,"num_bytes_recovered":1246,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":115536,"kb_used_data":25520,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188628144,"statfs":{"total":193273528320,"available":193155219456,"internally_reserved":0,"allocated":26132480,"data_stored":195520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-79354,"num_objects_misplaced":78318,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.945117"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":13624,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:27.259826+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:27.259826+0000","last_peered":"2026-02-22T20:51:27.259826+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:27.259826+0000","last_undegraded":"2026-02-22T20:51:27.259826+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14452,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1171,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":2829},{"shards":"2,3","objects":7226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":157672,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1229,"num_bytes_recovered":1246,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696442880,"data_stored":750752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051586,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182404,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57660,"kb_used_data":12652,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94314180,"statfs":{"total":96636764160,"available":96577720320,"internally_reserved":0,"allocated":12955648,"data_stored":97733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57876,"kb_used_data":12868,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94313964,"statfs":{"total":96636764160,"available":96577499136,"internally_reserved":0,"allocated":13176832,"data_stored":97787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13127680,"data_stored":3205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13127680,"data_stored":3205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:31.989 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:51:31.989 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10252, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188123+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:51:26.188123+0000', 'last_peered': '2026-02-22T20:51:26.188123+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:51:26.188123+0000', 'last_undegraded': '2026-02-22T20:51:26.188123+0000', 'last_fullsized': '2026-02-22T20:51:26.188123+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:31.989 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:51:31.989 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10378, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187383+0000', 'last_change': '2026-02-22T20:51:26.187530+0000', 'last_active': '2026-02-22T20:51:26.187383+0000', 'last_peered': '2026-02-22T20:51:26.187383+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:51:26.187383+0000', 'last_undegraded': '2026-02-22T20:51:26.187383+0000', 'last_fullsized': '2026-02-22T20:51:26.187383+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20566, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:31.989 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:51:31.989 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10169, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187844+0000', 'last_change': '2026-02-22T20:51:26.187976+0000', 'last_active': '2026-02-22T20:51:26.187844+0000', 'last_peered': '2026-02-22T20:51:26.187844+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:51:26.187844+0000', 'last_undegraded': '2026-02-22T20:51:26.187844+0000', 'last_fullsized': '2026-02-22T20:51:26.187844+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20202, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:31.989 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:51:31.989 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10495, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190360+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:51:26.190360+0000', 'last_peered': '2026-02-22T20:51:26.190360+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:51:26.190360+0000', 'last_undegraded': '2026-02-22T20:51:26.190360+0000', 'last_fullsized': '2026-02-22T20:51:26.190360+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:31.989 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:51:31.989 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 13624, 'reported_epoch': 31, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-22T20:51:27.259826+0000', 'last_change': '2026-02-22T20:51:24.247835+0000', 'last_active': '2026-02-22T20:51:27.259826+0000', 'last_peered': '2026-02-22T20:51:27.259826+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:51:15.334499+0000', 'last_became_peered': '2026-02-22T20:51:15.334499+0000', 'last_unstale': '2026-02-22T20:51:27.259826+0000', 'last_undegraded': '2026-02-22T20:51:27.259826+0000', 'last_fullsized': '2026-02-22T20:51:15.268613+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 14452, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1171, 'num_bytes_recovered': 1170, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 2829}, {'shards': '2,3', 'objects': 7226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:31.989 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:51:31.989 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10399, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188015+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:51:26.188015+0000', 'last_peered': '2026-02-22T20:51:26.188015+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:51:26.188015+0000', 'last_undegraded': '2026-02-22T20:51:26.188015+0000', 'last_fullsized': '2026-02-22T20:51:26.188015+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:31.990 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:51:31.990 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10301, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187898+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:51:26.187898+0000', 'last_peered': '2026-02-22T20:51:26.187898+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:51:26.187898+0000', 'last_undegraded': '2026-02-22T20:51:26.187898+0000', 'last_fullsized': '2026-02-22T20:51:26.187898+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:31.990 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:51:31.990 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10291, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190677+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:51:26.190677+0000', 'last_peered': '2026-02-22T20:51:26.190677+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:51:26.190677+0000', 'last_undegraded': '2026-02-22T20:51:26.190677+0000', 'last_fullsized': '2026-02-22T20:51:26.190677+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:31.990 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:51:32.208 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:32.208 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1108,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":31,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":118308864,"bytes_avail":193155219456,"bytes_total":193273528320,"misplaced_objects":157672,"misplaced_total":163508,"misplaced_ratio":0.96430755681679192,"recovering_objects_per_sec":74,"recovering_bytes_per_sec":74,"recovering_keys_per_sec":0,"num_objects_recovered":518,"num_bytes_recovered":518,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:51:32.222 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:51:32.222 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:32.403 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:32.404 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:32.417 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":581,"stamp":"2026-02-22T20:51:31.112915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":157672,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1229,"num_bytes_recovered":1246,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":115536,"kb_used_data":25520,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188628144,"statfs":{"total":193273528320,"available":193155219456,"internally_reserved":0,"allocated":26132480,"data_stored":195520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-79354,"num_objects_misplaced":78318,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.945117"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":13624,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:27.259826+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:27.259826+0000","last_peered":"2026-02-22T20:51:27.259826+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:27.259826+0000","last_undegraded":"2026-02-22T20:51:27.259826+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14452,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1171,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":2829},{"shards":"2,3","objects":7226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":157672,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1229,"num_bytes_recovered":1246,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696442880,"data_stored":750752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051586,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182404,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705890,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57660,"kb_used_data":12652,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94314180,"statfs":{"total":96636764160,"available":96577720320,"internally_reserved":0,"allocated":12955648,"data_stored":97733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57876,"kb_used_data":12868,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94313964,"statfs":{"total":96636764160,"available":96577499136,"internally_reserved":0,"allocated":13176832,"data_stored":97787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13127680,"data_stored":3205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13127680,"data_stored":3205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:35.419 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:35.603 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:35.604 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:35.617 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":583,"stamp":"2026-02-22T20:51:35.113431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":156604,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1763,"num_bytes_recovered":1780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":120052,"kb_used_data":30036,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188623628,"statfs":{"total":193273528320,"available":193150595072,"internally_reserved":0,"allocated":30756864,"data_stored":196649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-79354,"num_objects_misplaced":77250,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1052,"num_bytes_recovered":1052,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.940606"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":15226,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:32.267979+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:32.267979+0000","last_peered":"2026-02-22T20:51:32.267979+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:32.267979+0000","last_undegraded":"2026-02-22T20:51:32.267979+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13384,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1705,"num_bytes_recovered":1704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3363},{"shards":"2,3","objects":6692}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":156604,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1763,"num_bytes_recovered":1780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":701063168,"data_stored":751880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051587,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182405,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59888,"kb_used_data":14880,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94311952,"statfs":{"total":96636764160,"available":96575438848,"internally_reserved":0,"allocated":15237120,"data_stored":98290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60164,"kb_used_data":15156,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94311676,"statfs":{"total":96636764160,"available":96575156224,"internally_reserved":0,"allocated":15519744,"data_stored":98359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15437824,"data_stored":3769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15437824,"data_stored":3769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:35.617 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:35.800 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:35.800 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:35.813 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":583,"stamp":"2026-02-22T20:51:35.113431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":156604,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1763,"num_bytes_recovered":1780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":120052,"kb_used_data":30036,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188623628,"statfs":{"total":193273528320,"available":193150595072,"internally_reserved":0,"allocated":30756864,"data_stored":196649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-79354,"num_objects_misplaced":77250,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1052,"num_bytes_recovered":1052,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.940606"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":15226,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:32.267979+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:32.267979+0000","last_peered":"2026-02-22T20:51:32.267979+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:32.267979+0000","last_undegraded":"2026-02-22T20:51:32.267979+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13384,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1705,"num_bytes_recovered":1704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3363},{"shards":"2,3","objects":6692}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":156604,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1763,"num_bytes_recovered":1780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":701063168,"data_stored":751880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051587,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182405,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59888,"kb_used_data":14880,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94311952,"statfs":{"total":96636764160,"available":96575438848,"internally_reserved":0,"allocated":15237120,"data_stored":98290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60164,"kb_used_data":15156,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94311676,"statfs":{"total":96636764160,"available":96575156224,"internally_reserved":0,"allocated":15519744,"data_stored":98359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15437824,"data_stored":3769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15437824,"data_stored":3769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:35.814 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:51:35.815 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10252, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188123+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:51:26.188123+0000', 'last_peered': '2026-02-22T20:51:26.188123+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:51:26.188123+0000', 'last_undegraded': '2026-02-22T20:51:26.188123+0000', 'last_fullsized': '2026-02-22T20:51:26.188123+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:35.815 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:51:35.815 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10378, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187383+0000', 'last_change': '2026-02-22T20:51:26.187530+0000', 'last_active': '2026-02-22T20:51:26.187383+0000', 'last_peered': '2026-02-22T20:51:26.187383+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:51:26.187383+0000', 'last_undegraded': '2026-02-22T20:51:26.187383+0000', 'last_fullsized': '2026-02-22T20:51:26.187383+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20566, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:35.815 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:51:35.815 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10169, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187844+0000', 'last_change': '2026-02-22T20:51:26.187976+0000', 'last_active': '2026-02-22T20:51:26.187844+0000', 'last_peered': '2026-02-22T20:51:26.187844+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:51:26.187844+0000', 'last_undegraded': '2026-02-22T20:51:26.187844+0000', 'last_fullsized': '2026-02-22T20:51:26.187844+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20202, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:35.815 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:51:35.816 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10495, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190360+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:51:26.190360+0000', 'last_peered': '2026-02-22T20:51:26.190360+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:51:26.190360+0000', 'last_undegraded': '2026-02-22T20:51:26.190360+0000', 'last_fullsized': '2026-02-22T20:51:26.190360+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:35.816 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:51:35.816 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 15226, 'reported_epoch': 31, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-22T20:51:32.267979+0000', 'last_change': '2026-02-22T20:51:24.247835+0000', 'last_active': '2026-02-22T20:51:32.267979+0000', 'last_peered': '2026-02-22T20:51:32.267979+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:51:15.334499+0000', 'last_became_peered': '2026-02-22T20:51:15.334499+0000', 'last_unstale': '2026-02-22T20:51:32.267979+0000', 'last_undegraded': '2026-02-22T20:51:32.267979+0000', 'last_fullsized': '2026-02-22T20:51:15.268613+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 13384, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1705, 'num_bytes_recovered': 1704, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 3363}, {'shards': '2,3', 'objects': 6692}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:35.816 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:51:35.816 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10399, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188015+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:51:26.188015+0000', 'last_peered': '2026-02-22T20:51:26.188015+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:51:26.188015+0000', 'last_undegraded': '2026-02-22T20:51:26.188015+0000', 'last_fullsized': '2026-02-22T20:51:26.188015+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:35.816 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:51:35.816 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10301, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187898+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:51:26.187898+0000', 'last_peered': '2026-02-22T20:51:26.187898+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:51:26.187898+0000', 'last_undegraded': '2026-02-22T20:51:26.187898+0000', 'last_fullsized': '2026-02-22T20:51:26.187898+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:35.817 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:51:35.817 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10291, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190677+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:51:26.190677+0000', 'last_peered': '2026-02-22T20:51:26.190677+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:51:26.190677+0000', 'last_undegraded': '2026-02-22T20:51:26.190677+0000', 'last_fullsized': '2026-02-22T20:51:26.190677+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:35.817 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:51:36.037 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:36.037 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1112,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":31,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":122933248,"bytes_avail":193150595072,"bytes_total":193273528320,"misplaced_objects":156604,"misplaced_total":163508,"misplaced_ratio":0.95777576632336037,"recovering_objects_per_sec":105,"recovering_bytes_per_sec":105,"recovering_keys_per_sec":0,"num_objects_recovered":1052,"num_bytes_recovered":1052,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:51:36.052 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:51:36.052 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:36.235 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:36.235 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:36.248 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":583,"stamp":"2026-02-22T20:51:35.113431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":156604,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1763,"num_bytes_recovered":1780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":120052,"kb_used_data":30036,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188623628,"statfs":{"total":193273528320,"available":193150595072,"internally_reserved":0,"allocated":30756864,"data_stored":196649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-79354,"num_objects_misplaced":77250,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1052,"num_bytes_recovered":1052,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.940606"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":15226,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:32.267979+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:32.267979+0000","last_peered":"2026-02-22T20:51:32.267979+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:32.267979+0000","last_undegraded":"2026-02-22T20:51:32.267979+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13384,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1705,"num_bytes_recovered":1704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3363},{"shards":"2,3","objects":6692}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":156604,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1763,"num_bytes_recovered":1780,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":701063168,"data_stored":751880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051587,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182405,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59888,"kb_used_data":14880,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94311952,"statfs":{"total":96636764160,"available":96575438848,"internally_reserved":0,"allocated":15237120,"data_stored":98290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60164,"kb_used_data":15156,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94311676,"statfs":{"total":96636764160,"available":96575156224,"internally_reserved":0,"allocated":15519744,"data_stored":98359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15437824,"data_stored":3769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15437824,"data_stored":3769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:39.249 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:39.431 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:39.431 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:39.444 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":585,"stamp":"2026-02-22T20:51:39.113890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":155446,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":2359,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":120052,"kb_used_data":30036,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188623628,"statfs":{"total":193273528320,"available":193150595072,"internally_reserved":0,"allocated":30756864,"data_stored":196649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-79354,"num_objects_misplaced":76092,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1631,"num_bytes_recovered":1631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001505"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":16963,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:37.269040+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:37.269040+0000","last_peered":"2026-02-22T20:51:37.269040+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:37.269040+0000","last_undegraded":"2026-02-22T20:51:37.269040+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12226,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2284,"num_bytes_recovered":2283,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3942},{"shards":"2,3","objects":6113}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":155446,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":2359,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":701063168,"data_stored":751880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051588,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182406,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59888,"kb_used_data":14880,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94311952,"statfs":{"total":96636764160,"available":96575438848,"internally_reserved":0,"allocated":15237120,"data_stored":98290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60164,"kb_used_data":15156,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94311676,"statfs":{"total":96636764160,"available":96575156224,"internally_reserved":0,"allocated":15519744,"data_stored":98359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15437824,"data_stored":3769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15437824,"data_stored":3769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:39.446 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:39.627 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:39.627 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:39.640 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":585,"stamp":"2026-02-22T20:51:39.113890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":155446,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":2359,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":120052,"kb_used_data":30036,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188623628,"statfs":{"total":193273528320,"available":193150595072,"internally_reserved":0,"allocated":30756864,"data_stored":196649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-79354,"num_objects_misplaced":76092,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1631,"num_bytes_recovered":1631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001505"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":16963,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:37.269040+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:37.269040+0000","last_peered":"2026-02-22T20:51:37.269040+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:37.269040+0000","last_undegraded":"2026-02-22T20:51:37.269040+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12226,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2284,"num_bytes_recovered":2283,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3942},{"shards":"2,3","objects":6113}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":155446,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":2359,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":701063168,"data_stored":751880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051588,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182406,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59888,"kb_used_data":14880,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94311952,"statfs":{"total":96636764160,"available":96575438848,"internally_reserved":0,"allocated":15237120,"data_stored":98290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60164,"kb_used_data":15156,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94311676,"statfs":{"total":96636764160,"available":96575156224,"internally_reserved":0,"allocated":15519744,"data_stored":98359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15437824,"data_stored":3769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15437824,"data_stored":3769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:39.641 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:51:39.641 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10252, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188123+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:51:26.188123+0000', 'last_peered': '2026-02-22T20:51:26.188123+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:51:26.188123+0000', 'last_undegraded': '2026-02-22T20:51:26.188123+0000', 'last_fullsized': '2026-02-22T20:51:26.188123+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:39.641 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:51:39.641 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10378, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187383+0000', 'last_change': '2026-02-22T20:51:26.187530+0000', 'last_active': '2026-02-22T20:51:26.187383+0000', 'last_peered': '2026-02-22T20:51:26.187383+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:51:26.187383+0000', 'last_undegraded': '2026-02-22T20:51:26.187383+0000', 'last_fullsized': '2026-02-22T20:51:26.187383+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20566, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:39.641 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:51:39.641 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10169, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187844+0000', 'last_change': '2026-02-22T20:51:26.187976+0000', 'last_active': '2026-02-22T20:51:26.187844+0000', 'last_peered': '2026-02-22T20:51:26.187844+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:51:26.187844+0000', 'last_undegraded': '2026-02-22T20:51:26.187844+0000', 'last_fullsized': '2026-02-22T20:51:26.187844+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20202, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:39.641 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:51:39.641 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10495, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190360+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:51:26.190360+0000', 'last_peered': '2026-02-22T20:51:26.190360+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:51:26.190360+0000', 'last_undegraded': '2026-02-22T20:51:26.190360+0000', 'last_fullsized': '2026-02-22T20:51:26.190360+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:39.641 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:51:39.641 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 16963, 'reported_epoch': 31, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-22T20:51:37.269040+0000', 'last_change': '2026-02-22T20:51:24.247835+0000', 'last_active': '2026-02-22T20:51:37.269040+0000', 'last_peered': '2026-02-22T20:51:37.269040+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:51:15.334499+0000', 'last_became_peered': '2026-02-22T20:51:15.334499+0000', 'last_unstale': '2026-02-22T20:51:37.269040+0000', 'last_undegraded': '2026-02-22T20:51:37.269040+0000', 'last_fullsized': '2026-02-22T20:51:15.268613+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 12226, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2284, 'num_bytes_recovered': 2283, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 3942}, {'shards': '2,3', 'objects': 6113}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:39.642 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:51:39.642 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10399, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188015+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:51:26.188015+0000', 'last_peered': '2026-02-22T20:51:26.188015+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:51:26.188015+0000', 'last_undegraded': '2026-02-22T20:51:26.188015+0000', 'last_fullsized': '2026-02-22T20:51:26.188015+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:39.642 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:51:39.642 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10301, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187898+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:51:26.187898+0000', 'last_peered': '2026-02-22T20:51:26.187898+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:51:26.187898+0000', 'last_undegraded': '2026-02-22T20:51:26.187898+0000', 'last_fullsized': '2026-02-22T20:51:26.187898+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:39.642 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:51:39.642 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10291, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190677+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:51:26.190677+0000', 'last_peered': '2026-02-22T20:51:26.190677+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:51:26.190677+0000', 'last_undegraded': '2026-02-22T20:51:26.190677+0000', 'last_fullsized': '2026-02-22T20:51:26.190677+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:39.642 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:51:39.863 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:39.863 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1116,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":31,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":122933248,"bytes_avail":193150595072,"bytes_total":193273528320,"misplaced_objects":155446,"misplaced_total":163508,"misplaced_ratio":0.95069354404677442,"recovering_objects_per_sec":135,"recovering_bytes_per_sec":135,"recovering_keys_per_sec":0,"num_objects_recovered":1631,"num_bytes_recovered":1631,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:51:39.877 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:51:39.878 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:40.063 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:40.064 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:40.077 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":585,"stamp":"2026-02-22T20:51:39.113890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":155446,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":2359,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":120052,"kb_used_data":30036,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188623628,"statfs":{"total":193273528320,"available":193150595072,"internally_reserved":0,"allocated":30756864,"data_stored":196649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-79354,"num_objects_misplaced":76092,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1631,"num_bytes_recovered":1631,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001505"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":16963,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:37.269040+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:37.269040+0000","last_peered":"2026-02-22T20:51:37.269040+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:37.269040+0000","last_undegraded":"2026-02-22T20:51:37.269040+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12226,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2284,"num_bytes_recovered":2283,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":3942},{"shards":"2,3","objects":6113}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":155446,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":2359,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":701063168,"data_stored":751880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051588,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182406,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59888,"kb_used_data":14880,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94311952,"statfs":{"total":96636764160,"available":96575438848,"internally_reserved":0,"allocated":15237120,"data_stored":98290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60164,"kb_used_data":15156,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94311676,"statfs":{"total":96636764160,"available":96575156224,"internally_reserved":0,"allocated":15519744,"data_stored":98359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15437824,"data_stored":3769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15437824,"data_stored":3769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:43.079 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:43.265 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:43.265 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:43.279 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":587,"stamp":"2026-02-22T20:51:43.114356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":154170,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":2997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":125308,"kb_used_data":35292,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188618372,"statfs":{"total":193273528320,"available":193145212928,"internally_reserved":0,"allocated":36139008,"data_stored":197963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3502,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1751,"num_bytes_recovered":1751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001440"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":18877,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:42.274925+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:42.274925+0000","last_peered":"2026-02-22T20:51:42.274925+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:42.274925+0000","last_undegraded":"2026-02-22T20:51:42.274925+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10950,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":2921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":4580},{"shards":"2,3","objects":5475}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":154170,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":2997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":705986560,"data_stored":753082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051588,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182407,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62720,"kb_used_data":17712,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94309120,"statfs":{"total":96636764160,"available":96572538880,"internally_reserved":0,"allocated":18137088,"data_stored":98998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62588,"kb_used_data":17580,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94309252,"statfs":{"total":96636764160,"available":96572674048,"internally_reserved":0,"allocated":18001920,"data_stored":98965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17899520,"data_stored":4370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17899520,"data_stored":4370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:43.280 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:43.463 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:43.463 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:43.476 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":587,"stamp":"2026-02-22T20:51:43.114356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":154170,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":2997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":125308,"kb_used_data":35292,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188618372,"statfs":{"total":193273528320,"available":193145212928,"internally_reserved":0,"allocated":36139008,"data_stored":197963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3502,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1751,"num_bytes_recovered":1751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001440"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":18877,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:42.274925+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:42.274925+0000","last_peered":"2026-02-22T20:51:42.274925+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:42.274925+0000","last_undegraded":"2026-02-22T20:51:42.274925+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10950,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":2921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":4580},{"shards":"2,3","objects":5475}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":154170,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":2997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":705986560,"data_stored":753082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051588,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182407,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62720,"kb_used_data":17712,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94309120,"statfs":{"total":96636764160,"available":96572538880,"internally_reserved":0,"allocated":18137088,"data_stored":98998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62588,"kb_used_data":17580,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94309252,"statfs":{"total":96636764160,"available":96572674048,"internally_reserved":0,"allocated":18001920,"data_stored":98965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17899520,"data_stored":4370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17899520,"data_stored":4370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:43.477 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:51:43.477 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10252, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188123+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:51:26.188123+0000', 'last_peered': '2026-02-22T20:51:26.188123+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:51:26.188123+0000', 'last_undegraded': '2026-02-22T20:51:26.188123+0000', 'last_fullsized': '2026-02-22T20:51:26.188123+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:43.477 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:51:43.477 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10378, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187383+0000', 'last_change': '2026-02-22T20:51:26.187530+0000', 'last_active': '2026-02-22T20:51:26.187383+0000', 'last_peered': '2026-02-22T20:51:26.187383+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:51:26.187383+0000', 'last_undegraded': '2026-02-22T20:51:26.187383+0000', 'last_fullsized': '2026-02-22T20:51:26.187383+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20566, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:43.478 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:51:43.478 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10169, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187844+0000', 'last_change': '2026-02-22T20:51:26.187976+0000', 'last_active': '2026-02-22T20:51:26.187844+0000', 'last_peered': '2026-02-22T20:51:26.187844+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:51:26.187844+0000', 'last_undegraded': '2026-02-22T20:51:26.187844+0000', 'last_fullsized': '2026-02-22T20:51:26.187844+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20202, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:43.478 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:51:43.478 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10495, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190360+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:51:26.190360+0000', 'last_peered': '2026-02-22T20:51:26.190360+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:51:26.190360+0000', 'last_undegraded': '2026-02-22T20:51:26.190360+0000', 'last_fullsized': '2026-02-22T20:51:26.190360+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:43.478 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:51:43.478 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 18877, 'reported_epoch': 31, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-22T20:51:42.274925+0000', 'last_change': '2026-02-22T20:51:24.247835+0000', 'last_active': '2026-02-22T20:51:42.274925+0000', 'last_peered': '2026-02-22T20:51:42.274925+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:51:15.334499+0000', 'last_became_peered': '2026-02-22T20:51:15.334499+0000', 'last_unstale': '2026-02-22T20:51:42.274925+0000', 'last_undegraded': '2026-02-22T20:51:42.274925+0000', 'last_fullsized': '2026-02-22T20:51:15.268613+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 10950, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2922, 'num_bytes_recovered': 2921, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 4580}, {'shards': '2,3', 'objects': 5475}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:43.478 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:51:43.478 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10399, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188015+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:51:26.188015+0000', 'last_peered': '2026-02-22T20:51:26.188015+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:51:26.188015+0000', 'last_undegraded': '2026-02-22T20:51:26.188015+0000', 'last_fullsized': '2026-02-22T20:51:26.188015+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:43.478 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:51:43.478 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10301, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187898+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:51:26.187898+0000', 'last_peered': '2026-02-22T20:51:26.187898+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:51:26.187898+0000', 'last_undegraded': '2026-02-22T20:51:26.187898+0000', 'last_fullsized': '2026-02-22T20:51:26.187898+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:43.478 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:51:43.478 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10291, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190677+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:51:26.190677+0000', 'last_peered': '2026-02-22T20:51:26.190677+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:51:26.190677+0000', 'last_undegraded': '2026-02-22T20:51:26.190677+0000', 'last_fullsized': '2026-02-22T20:51:26.190677+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:43.478 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:51:43.700 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:43.700 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1120,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":31,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":128315392,"bytes_avail":193145212928,"bytes_total":193273528320,"misplaced_objects":154170,"misplaced_total":163508,"misplaced_ratio":0.94288964454338631,"recovering_objects_per_sec":145,"recovering_bytes_per_sec":145,"recovering_keys_per_sec":0,"num_objects_recovered":1751,"num_bytes_recovered":1751,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:51:43.715 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:51:43.715 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:43.899 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:43.900 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:43.914 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":587,"stamp":"2026-02-22T20:51:43.114356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":154170,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":2997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":125308,"kb_used_data":35292,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188618372,"statfs":{"total":193273528320,"available":193145212928,"internally_reserved":0,"allocated":36139008,"data_stored":197963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3502,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1751,"num_bytes_recovered":1751,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001440"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":18877,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:42.274925+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:42.274925+0000","last_peered":"2026-02-22T20:51:42.274925+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:42.274925+0000","last_undegraded":"2026-02-22T20:51:42.274925+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10950,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":2921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":4580},{"shards":"2,3","objects":5475}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":154170,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":2997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":705986560,"data_stored":753082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051588,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182407,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62720,"kb_used_data":17712,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94309120,"statfs":{"total":96636764160,"available":96572538880,"internally_reserved":0,"allocated":18137088,"data_stored":98998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62588,"kb_used_data":17580,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94309252,"statfs":{"total":96636764160,"available":96572674048,"internally_reserved":0,"allocated":18001920,"data_stored":98965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17899520,"data_stored":4370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17899520,"data_stored":4370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:46.917 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:47.100 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:47.100 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:47.113 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":588,"stamp":"2026-02-22T20:51:45.114611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":154170,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":2997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":130656,"kb_used_data":40640,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188613024,"statfs":{"total":193273528320,"available":193139736576,"internally_reserved":0,"allocated":41615360,"data_stored":199300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2434,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1217,"num_bytes_recovered":1217,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001468"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":18877,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:42.274925+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:42.274925+0000","last_peered":"2026-02-22T20:51:42.274925+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:42.274925+0000","last_undegraded":"2026-02-22T20:51:42.274925+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10950,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":2921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":4580},{"shards":"2,3","objects":5475}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":154170,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":2997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":711434240,"data_stored":754412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051589,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182407,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65372,"kb_used_data":20364,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94306468,"statfs":{"total":96636764160,"available":96569823232,"internally_reserved":0,"allocated":20852736,"data_stored":99661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65284,"kb_used_data":20276,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94306556,"statfs":{"total":96636764160,"available":96569913344,"internally_reserved":0,"allocated":20762624,"data_stored":99639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20623360,"data_stored":5035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20623360,"data_stored":5035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:47.114 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:47.297 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:47.298 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:47.311 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":589,"stamp":"2026-02-22T20:51:47.114832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":154170,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":2997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":130656,"kb_used_data":40640,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188613024,"statfs":{"total":193273528320,"available":193139736576,"internally_reserved":0,"allocated":41615360,"data_stored":199300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2434,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1217,"num_bytes_recovered":1217,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":18877,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:42.274925+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:42.274925+0000","last_peered":"2026-02-22T20:51:42.274925+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:42.274925+0000","last_undegraded":"2026-02-22T20:51:42.274925+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10950,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":2921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":4580},{"shards":"2,3","objects":5475}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":154170,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":2997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":711434240,"data_stored":754412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051589,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182407,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65372,"kb_used_data":20364,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94306468,"statfs":{"total":96636764160,"available":96569823232,"internally_reserved":0,"allocated":20852736,"data_stored":99661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65284,"kb_used_data":20276,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94306556,"statfs":{"total":96636764160,"available":96569913344,"internally_reserved":0,"allocated":20762624,"data_stored":99639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20623360,"data_stored":5035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20623360,"data_stored":5035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:47.312 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:51:47.312 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10252, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188123+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:51:26.188123+0000', 'last_peered': '2026-02-22T20:51:26.188123+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:51:26.188123+0000', 'last_undegraded': '2026-02-22T20:51:26.188123+0000', 'last_fullsized': '2026-02-22T20:51:26.188123+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:47.313 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:51:47.313 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10378, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187383+0000', 'last_change': '2026-02-22T20:51:26.187530+0000', 'last_active': '2026-02-22T20:51:26.187383+0000', 'last_peered': '2026-02-22T20:51:26.187383+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:51:26.187383+0000', 'last_undegraded': '2026-02-22T20:51:26.187383+0000', 'last_fullsized': '2026-02-22T20:51:26.187383+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20566, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:47.313 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:51:47.313 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10169, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187844+0000', 'last_change': '2026-02-22T20:51:26.187976+0000', 'last_active': '2026-02-22T20:51:26.187844+0000', 'last_peered': '2026-02-22T20:51:26.187844+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:51:26.187844+0000', 'last_undegraded': '2026-02-22T20:51:26.187844+0000', 'last_fullsized': '2026-02-22T20:51:26.187844+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20202, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:47.313 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:51:47.313 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10495, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190360+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:51:26.190360+0000', 'last_peered': '2026-02-22T20:51:26.190360+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:51:26.190360+0000', 'last_undegraded': '2026-02-22T20:51:26.190360+0000', 'last_fullsized': '2026-02-22T20:51:26.190360+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:47.313 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:51:47.314 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 18877, 'reported_epoch': 31, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-22T20:51:42.274925+0000', 'last_change': '2026-02-22T20:51:24.247835+0000', 'last_active': '2026-02-22T20:51:42.274925+0000', 'last_peered': '2026-02-22T20:51:42.274925+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:51:15.334499+0000', 'last_became_peered': '2026-02-22T20:51:15.334499+0000', 'last_unstale': '2026-02-22T20:51:42.274925+0000', 'last_undegraded': '2026-02-22T20:51:42.274925+0000', 'last_fullsized': '2026-02-22T20:51:15.268613+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 10950, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2922, 'num_bytes_recovered': 2921, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 4580}, {'shards': '2,3', 'objects': 5475}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:47.314 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:51:47.314 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10399, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188015+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:51:26.188015+0000', 'last_peered': '2026-02-22T20:51:26.188015+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:51:26.188015+0000', 'last_undegraded': '2026-02-22T20:51:26.188015+0000', 'last_fullsized': '2026-02-22T20:51:26.188015+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:47.314 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:51:47.314 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10301, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187898+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:51:26.187898+0000', 'last_peered': '2026-02-22T20:51:26.187898+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:51:26.187898+0000', 'last_undegraded': '2026-02-22T20:51:26.187898+0000', 'last_fullsized': '2026-02-22T20:51:26.187898+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:47.314 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:51:47.314 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10291, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190677+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:51:26.190677+0000', 'last_peered': '2026-02-22T20:51:26.190677+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:51:26.190677+0000', 'last_undegraded': '2026-02-22T20:51:26.190677+0000', 'last_fullsized': '2026-02-22T20:51:26.190677+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:47.315 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:51:47.534 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:47.535 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1124,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":31,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":133791744,"bytes_avail":193139736576,"bytes_total":193273528320,"misplaced_objects":154170,"misplaced_total":163508,"misplaced_ratio":0.94288964454338631,"recovering_objects_per_sec":101,"recovering_bytes_per_sec":101,"recovering_keys_per_sec":0,"num_objects_recovered":1217,"num_bytes_recovered":1217,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:51:47.549 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:51:47.550 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:47.731 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:47.732 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:47.745 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":589,"stamp":"2026-02-22T20:51:47.114832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":154170,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":2997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":130656,"kb_used_data":40640,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188613024,"statfs":{"total":193273528320,"available":193139736576,"internally_reserved":0,"allocated":41615360,"data_stored":199300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2434,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1217,"num_bytes_recovered":1217,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001400"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":18877,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:42.274925+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:42.274925+0000","last_peered":"2026-02-22T20:51:42.274925+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:42.274925+0000","last_undegraded":"2026-02-22T20:51:42.274925+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10950,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2922,"num_bytes_recovered":2921,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":4580},{"shards":"2,3","objects":5475}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":154170,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":2997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":711434240,"data_stored":754412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051589,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182407,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65372,"kb_used_data":20364,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94306468,"statfs":{"total":96636764160,"available":96569823232,"internally_reserved":0,"allocated":20852736,"data_stored":99661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65284,"kb_used_data":20276,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94306556,"statfs":{"total":96636764160,"available":96569913344,"internally_reserved":0,"allocated":20762624,"data_stored":99639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20623360,"data_stored":5035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20623360,"data_stored":5035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:50.748 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:50.932 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:50.933 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:50.947 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":590,"stamp":"2026-02-22T20:51:49.115033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":152790,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3670,"num_bytes_recovered":3687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":130656,"kb_used_data":40640,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188613024,"statfs":{"total":193273528320,"available":193139736576,"internally_reserved":0,"allocated":41615360,"data_stored":199300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3814,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1907,"num_bytes_recovered":1907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001360"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":20947,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:47.279015+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:47.279015+0000","last_peered":"2026-02-22T20:51:47.279015+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:47.279015+0000","last_undegraded":"2026-02-22T20:51:47.279015+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9570,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":3611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":5270},{"shards":"2,3","objects":4785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":152790,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3670,"num_bytes_recovered":3687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":711434240,"data_stored":754412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051590,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182408,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65372,"kb_used_data":20364,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94306468,"statfs":{"total":96636764160,"available":96569823232,"internally_reserved":0,"allocated":20852736,"data_stored":99661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65284,"kb_used_data":20276,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94306556,"statfs":{"total":96636764160,"available":96569913344,"internally_reserved":0,"allocated":20762624,"data_stored":99639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20623360,"data_stored":5035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20623360,"data_stored":5035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:50.947 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:51.131 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:51.132 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:51.147 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":591,"stamp":"2026-02-22T20:51:51.115276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":152790,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3670,"num_bytes_recovered":3687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":136600,"kb_used_data":46584,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188607080,"statfs":{"total":193273528320,"available":193133649920,"internally_reserved":0,"allocated":47702016,"data_stored":200786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2656,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1328,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001385"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":20947,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:47.279015+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:47.279015+0000","last_peered":"2026-02-22T20:51:47.279015+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:47.279015+0000","last_undegraded":"2026-02-22T20:51:47.279015+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9570,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":3611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":5270},{"shards":"2,3","objects":4785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":152790,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3670,"num_bytes_recovered":3687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":717537280,"data_stored":755902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051590,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182408,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68296,"kb_used_data":23288,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94303544,"statfs":{"total":96636764160,"available":96566829056,"internally_reserved":0,"allocated":23846912,"data_stored":100392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68304,"kb_used_data":23296,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94303536,"statfs":{"total":96636764160,"available":96566820864,"internally_reserved":0,"allocated":23855104,"data_stored":100394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23674880,"data_stored":5780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23674880,"data_stored":5780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:51.147 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:51:51.147 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10252, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188123+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:51:26.188123+0000', 'last_peered': '2026-02-22T20:51:26.188123+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:51:26.188123+0000', 'last_undegraded': '2026-02-22T20:51:26.188123+0000', 'last_fullsized': '2026-02-22T20:51:26.188123+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:51.147 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:51:51.147 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10378, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187383+0000', 'last_change': '2026-02-22T20:51:26.187530+0000', 'last_active': '2026-02-22T20:51:26.187383+0000', 'last_peered': '2026-02-22T20:51:26.187383+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:51:26.187383+0000', 'last_undegraded': '2026-02-22T20:51:26.187383+0000', 'last_fullsized': '2026-02-22T20:51:26.187383+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20566, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:51.147 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:51:51.147 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10169, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187844+0000', 'last_change': '2026-02-22T20:51:26.187976+0000', 'last_active': '2026-02-22T20:51:26.187844+0000', 'last_peered': '2026-02-22T20:51:26.187844+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:51:26.187844+0000', 'last_undegraded': '2026-02-22T20:51:26.187844+0000', 'last_fullsized': '2026-02-22T20:51:26.187844+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20202, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:51.147 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:51:51.147 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10495, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190360+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:51:26.190360+0000', 'last_peered': '2026-02-22T20:51:26.190360+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:51:26.190360+0000', 'last_undegraded': '2026-02-22T20:51:26.190360+0000', 'last_fullsized': '2026-02-22T20:51:26.190360+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:51.147 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:51:51.148 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 20947, 'reported_epoch': 31, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-22T20:51:47.279015+0000', 'last_change': '2026-02-22T20:51:24.247835+0000', 'last_active': '2026-02-22T20:51:47.279015+0000', 'last_peered': '2026-02-22T20:51:47.279015+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:51:15.334499+0000', 'last_became_peered': '2026-02-22T20:51:15.334499+0000', 'last_unstale': '2026-02-22T20:51:47.279015+0000', 'last_undegraded': '2026-02-22T20:51:47.279015+0000', 'last_fullsized': '2026-02-22T20:51:15.268613+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 9570, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3612, 'num_bytes_recovered': 3611, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 5270}, {'shards': '2,3', 'objects': 4785}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:51.148 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:51:51.148 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10399, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188015+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:51:26.188015+0000', 'last_peered': '2026-02-22T20:51:26.188015+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:51:26.188015+0000', 'last_undegraded': '2026-02-22T20:51:26.188015+0000', 'last_fullsized': '2026-02-22T20:51:26.188015+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:51.148 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:51:51.148 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10301, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187898+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:51:26.187898+0000', 'last_peered': '2026-02-22T20:51:26.187898+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:51:26.187898+0000', 'last_undegraded': '2026-02-22T20:51:26.187898+0000', 'last_fullsized': '2026-02-22T20:51:26.187898+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:51.148 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:51:51.148 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10291, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190677+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:51:26.190677+0000', 'last_peered': '2026-02-22T20:51:26.190677+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:51:26.190677+0000', 'last_undegraded': '2026-02-22T20:51:26.190677+0000', 'last_fullsized': '2026-02-22T20:51:26.190677+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:51.148 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:51:51.380 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:51.380 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1128,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":31,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":139878400,"bytes_avail":193133649920,"bytes_total":193273528320,"misplaced_objects":152790,"misplaced_total":163508,"misplaced_ratio":0.93444969053501969,"recovering_objects_per_sec":110,"recovering_bytes_per_sec":110,"recovering_keys_per_sec":0,"num_objects_recovered":1328,"num_bytes_recovered":1328,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:51:51.393 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:51:51.394 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:51.574 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:51.575 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:51.590 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":591,"stamp":"2026-02-22T20:51:51.115276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":152790,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3670,"num_bytes_recovered":3687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":136600,"kb_used_data":46584,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188607080,"statfs":{"total":193273528320,"available":193133649920,"internally_reserved":0,"allocated":47702016,"data_stored":200786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2656,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1328,"num_bytes_recovered":1328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001385"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":20947,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:47.279015+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:47.279015+0000","last_peered":"2026-02-22T20:51:47.279015+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:47.279015+0000","last_undegraded":"2026-02-22T20:51:47.279015+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9570,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":3611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":5270},{"shards":"2,3","objects":4785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":152790,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3670,"num_bytes_recovered":3687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":717537280,"data_stored":755902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051590,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182408,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68296,"kb_used_data":23288,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94303544,"statfs":{"total":96636764160,"available":96566829056,"internally_reserved":0,"allocated":23846912,"data_stored":100392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68304,"kb_used_data":23296,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94303536,"statfs":{"total":96636764160,"available":96566820864,"internally_reserved":0,"allocated":23855104,"data_stored":100394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23674880,"data_stored":5780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23674880,"data_stored":5780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:54.591 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:54.784 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:54.784 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:54.798 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":592,"stamp":"2026-02-22T20:51:53.115528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":151216,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":136600,"kb_used_data":46584,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188607080,"statfs":{"total":193273528320,"available":193133649920,"internally_reserved":0,"allocated":47702016,"data_stored":200786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-4230,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2115,"num_bytes_recovered":2115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001416"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":23308,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:52.281048+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:52.281048+0000","last_peered":"2026-02-22T20:51:52.281048+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:52.281048+0000","last_undegraded":"2026-02-22T20:51:52.281048+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7996,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4399,"num_bytes_recovered":4398,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6057},{"shards":"2,3","objects":3998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":151216,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":717537280,"data_stored":755902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051590,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182409,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68296,"kb_used_data":23288,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94303544,"statfs":{"total":96636764160,"available":96566829056,"internally_reserved":0,"allocated":23846912,"data_stored":100392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68304,"kb_used_data":23296,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94303536,"statfs":{"total":96636764160,"available":96566820864,"internally_reserved":0,"allocated":23855104,"data_stored":100394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23674880,"data_stored":5780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23674880,"data_stored":5780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:54.799 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:54.982 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:54.982 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:54.995 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":592,"stamp":"2026-02-22T20:51:53.115528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":151216,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":136600,"kb_used_data":46584,"kb_used_omap":35,"kb_used_meta":89948,"kb_avail":188607080,"statfs":{"total":193273528320,"available":193133649920,"internally_reserved":0,"allocated":47702016,"data_stored":200786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":92107662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-4230,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2115,"num_bytes_recovered":2115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001416"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":23308,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:52.281048+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:52.281048+0000","last_peered":"2026-02-22T20:51:52.281048+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:52.281048+0000","last_undegraded":"2026-02-22T20:51:52.281048+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7996,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4399,"num_bytes_recovered":4398,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6057},{"shards":"2,3","objects":3998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":151216,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":717537280,"data_stored":755902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051590,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182409,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68296,"kb_used_data":23288,"kb_used_omap":18,"kb_used_meta":44973,"kb_avail":94303544,"statfs":{"total":96636764160,"available":96566829056,"internally_reserved":0,"allocated":23846912,"data_stored":100392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":46053180},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68304,"kb_used_data":23296,"kb_used_omap":16,"kb_used_meta":44975,"kb_avail":94303536,"statfs":{"total":96636764160,"available":96566820864,"internally_reserved":0,"allocated":23855104,"data_stored":100394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":46054482},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23674880,"data_stored":5780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23674880,"data_stored":5780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:54.997 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:51:54.997 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10252, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188123+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:51:26.188123+0000', 'last_peered': '2026-02-22T20:51:26.188123+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:51:26.188123+0000', 'last_undegraded': '2026-02-22T20:51:26.188123+0000', 'last_fullsized': '2026-02-22T20:51:26.188123+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:54.997 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:51:54.997 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10378, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187383+0000', 'last_change': '2026-02-22T20:51:26.187530+0000', 'last_active': '2026-02-22T20:51:26.187383+0000', 'last_peered': '2026-02-22T20:51:26.187383+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:51:26.187383+0000', 'last_undegraded': '2026-02-22T20:51:26.187383+0000', 'last_fullsized': '2026-02-22T20:51:26.187383+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20566, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:54.997 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:51:54.998 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10169, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187844+0000', 'last_change': '2026-02-22T20:51:26.187976+0000', 'last_active': '2026-02-22T20:51:26.187844+0000', 'last_peered': '2026-02-22T20:51:26.187844+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:51:26.187844+0000', 'last_undegraded': '2026-02-22T20:51:26.187844+0000', 'last_fullsized': '2026-02-22T20:51:26.187844+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20202, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:54.998 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:51:54.998 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10495, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190360+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:51:26.190360+0000', 'last_peered': '2026-02-22T20:51:26.190360+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:51:26.190360+0000', 'last_undegraded': '2026-02-22T20:51:26.190360+0000', 'last_fullsized': '2026-02-22T20:51:26.190360+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:54.998 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:51:54.998 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 23308, 'reported_epoch': 31, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-22T20:51:52.281048+0000', 'last_change': '2026-02-22T20:51:24.247835+0000', 'last_active': '2026-02-22T20:51:52.281048+0000', 'last_peered': '2026-02-22T20:51:52.281048+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:51:15.334499+0000', 'last_became_peered': '2026-02-22T20:51:15.334499+0000', 'last_unstale': '2026-02-22T20:51:52.281048+0000', 'last_undegraded': '2026-02-22T20:51:52.281048+0000', 'last_fullsized': '2026-02-22T20:51:15.268613+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 7996, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4399, 'num_bytes_recovered': 4398, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 6057}, {'shards': '2,3', 'objects': 3998}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:54.998 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:51:54.998 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10399, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188015+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:51:26.188015+0000', 'last_peered': '2026-02-22T20:51:26.188015+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:51:26.188015+0000', 'last_undegraded': '2026-02-22T20:51:26.188015+0000', 'last_fullsized': '2026-02-22T20:51:26.188015+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:54.999 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:51:54.999 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10301, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187898+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:51:26.187898+0000', 'last_peered': '2026-02-22T20:51:26.187898+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:51:26.187898+0000', 'last_undegraded': '2026-02-22T20:51:26.187898+0000', 'last_fullsized': '2026-02-22T20:51:26.187898+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:54.999 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:51:54.999 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10291, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190677+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:51:26.190677+0000', 'last_peered': '2026-02-22T20:51:26.190677+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:51:26.190677+0000', 'last_undegraded': '2026-02-22T20:51:26.190677+0000', 'last_fullsized': '2026-02-22T20:51:26.190677+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:54.999 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:51:55.225 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:55.225 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1131,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":31,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":220774400,"bytes_avail":193052753920,"bytes_total":193273528320,"misplaced_objects":151216,"misplaced_total":163508,"misplaced_ratio":0.92482325023852041,"recovering_objects_per_sec":123,"recovering_bytes_per_sec":123,"recovering_keys_per_sec":0,"num_objects_recovered":1477,"num_bytes_recovered":1477,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:51:55.240 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:51:55.240 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:55.428 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:55.429 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:55.442 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":593,"stamp":"2026-02-22T20:51:55.115792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":151216,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":215600,"kb_used_data":53392,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188528080,"statfs":{"total":193273528320,"available":193052753920,"internally_reserved":0,"allocated":54673408,"data_stored":202488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2954,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1477,"num_bytes_recovered":1477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001436"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":23308,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:52.281048+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:52.281048+0000","last_peered":"2026-02-22T20:51:52.281048+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:52.281048+0000","last_undegraded":"2026-02-22T20:51:52.281048+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7996,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4399,"num_bytes_recovered":4398,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6057},{"shards":"2,3","objects":3998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":151216,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724516864,"data_stored":757606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051591,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182409,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107836,"kb_used_data":26732,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94264004,"statfs":{"total":96636764160,"available":96526340096,"internally_reserved":0,"allocated":27373568,"data_stored":101253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107764,"kb_used_data":26660,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94264076,"statfs":{"total":96636764160,"available":96526413824,"internally_reserved":0,"allocated":27299840,"data_stored":101235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27164672,"data_stored":6632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27164672,"data_stored":6632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:58.444 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:58.632 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:58.633 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:58.645 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":594,"stamp":"2026-02-22T20:51:57.116045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":151216,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":215600,"kb_used_data":53392,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188528080,"statfs":{"total":193273528320,"available":193052753920,"internally_reserved":0,"allocated":54673408,"data_stored":202488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2954,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1477,"num_bytes_recovered":1477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":23308,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:52.281048+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:52.281048+0000","last_peered":"2026-02-22T20:51:52.281048+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:52.281048+0000","last_undegraded":"2026-02-22T20:51:52.281048+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7996,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4399,"num_bytes_recovered":4398,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6057},{"shards":"2,3","objects":3998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":151216,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724516864,"data_stored":757606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051591,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182409,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107836,"kb_used_data":26732,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94264004,"statfs":{"total":96636764160,"available":96526340096,"internally_reserved":0,"allocated":27373568,"data_stored":101253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107764,"kb_used_data":26660,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94264076,"statfs":{"total":96636764160,"available":96526413824,"internally_reserved":0,"allocated":27299840,"data_stored":101235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27164672,"data_stored":6632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27164672,"data_stored":6632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:58.646 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:58.835 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:58.836 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:58.849 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":594,"stamp":"2026-02-22T20:51:57.116045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":151216,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":215600,"kb_used_data":53392,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188528080,"statfs":{"total":193273528320,"available":193052753920,"internally_reserved":0,"allocated":54673408,"data_stored":202488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2954,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1477,"num_bytes_recovered":1477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001433"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":23308,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:52.281048+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:52.281048+0000","last_peered":"2026-02-22T20:51:52.281048+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:52.281048+0000","last_undegraded":"2026-02-22T20:51:52.281048+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7996,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4399,"num_bytes_recovered":4398,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6057},{"shards":"2,3","objects":3998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":151216,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4474,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724516864,"data_stored":757606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051591,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182409,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107836,"kb_used_data":26732,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94264004,"statfs":{"total":96636764160,"available":96526340096,"internally_reserved":0,"allocated":27373568,"data_stored":101253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107764,"kb_used_data":26660,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94264076,"statfs":{"total":96636764160,"available":96526413824,"internally_reserved":0,"allocated":27299840,"data_stored":101235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27164672,"data_stored":6632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27164672,"data_stored":6632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:51:58.850 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:51:58.850 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10252, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188123+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:51:26.188123+0000', 'last_peered': '2026-02-22T20:51:26.188123+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:51:26.188123+0000', 'last_undegraded': '2026-02-22T20:51:26.188123+0000', 'last_fullsized': '2026-02-22T20:51:26.188123+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:58.850 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:51:58.850 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10378, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187383+0000', 'last_change': '2026-02-22T20:51:26.187530+0000', 'last_active': '2026-02-22T20:51:26.187383+0000', 'last_peered': '2026-02-22T20:51:26.187383+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:51:26.187383+0000', 'last_undegraded': '2026-02-22T20:51:26.187383+0000', 'last_fullsized': '2026-02-22T20:51:26.187383+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20566, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:58.851 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:51:58.851 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10169, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187844+0000', 'last_change': '2026-02-22T20:51:26.187976+0000', 'last_active': '2026-02-22T20:51:26.187844+0000', 'last_peered': '2026-02-22T20:51:26.187844+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:51:26.187844+0000', 'last_undegraded': '2026-02-22T20:51:26.187844+0000', 'last_fullsized': '2026-02-22T20:51:26.187844+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20202, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:58.851 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:51:58.851 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10495, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190360+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:51:26.190360+0000', 'last_peered': '2026-02-22T20:51:26.190360+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:51:26.190360+0000', 'last_undegraded': '2026-02-22T20:51:26.190360+0000', 'last_fullsized': '2026-02-22T20:51:26.190360+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:58.851 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:51:58.851 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 23308, 'reported_epoch': 31, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-22T20:51:52.281048+0000', 'last_change': '2026-02-22T20:51:24.247835+0000', 'last_active': '2026-02-22T20:51:52.281048+0000', 'last_peered': '2026-02-22T20:51:52.281048+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:51:15.334499+0000', 'last_became_peered': '2026-02-22T20:51:15.334499+0000', 'last_unstale': '2026-02-22T20:51:52.281048+0000', 'last_undegraded': '2026-02-22T20:51:52.281048+0000', 'last_fullsized': '2026-02-22T20:51:15.268613+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 7996, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4399, 'num_bytes_recovered': 4398, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 6057}, {'shards': '2,3', 'objects': 3998}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:58.851 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:51:58.852 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10399, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188015+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:51:26.188015+0000', 'last_peered': '2026-02-22T20:51:26.188015+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:51:26.188015+0000', 'last_undegraded': '2026-02-22T20:51:26.188015+0000', 'last_fullsized': '2026-02-22T20:51:26.188015+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:58.852 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:51:58.852 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10301, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187898+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:51:26.187898+0000', 'last_peered': '2026-02-22T20:51:26.187898+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:51:26.187898+0000', 'last_undegraded': '2026-02-22T20:51:26.187898+0000', 'last_fullsized': '2026-02-22T20:51:26.187898+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:58.852 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:51:58.852 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10291, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190677+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:51:26.190677+0000', 'last_peered': '2026-02-22T20:51:26.190677+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:51:26.190677+0000', 'last_undegraded': '2026-02-22T20:51:26.190677+0000', 'last_fullsized': '2026-02-22T20:51:26.190677+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:51:58.852 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:51:59.073 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:59.073 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1135,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":31,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":220774400,"bytes_avail":193052753920,"bytes_total":193273528320,"misplaced_objects":151216,"misplaced_total":163508,"misplaced_ratio":0.92482325023852041,"recovering_objects_per_sec":123,"recovering_bytes_per_sec":123,"recovering_keys_per_sec":0,"num_objects_recovered":1477,"num_bytes_recovered":1477,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:51:59.087 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:51:59.087 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:51:59.276 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:51:59.276 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:51:59.293 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":595,"stamp":"2026-02-22T20:51:59.116261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":149374,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5378,"num_bytes_recovered":5395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":215600,"kb_used_data":53392,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188528080,"statfs":{"total":193273528320,"available":193052753920,"internally_reserved":0,"allocated":54673408,"data_stored":202488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-4796,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2398,"num_bytes_recovered":2398,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001429"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":26071,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:57.280472+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:57.280472+0000","last_peered":"2026-02-22T20:51:57.280472+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:57.280472+0000","last_undegraded":"2026-02-22T20:51:57.280472+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6154,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5320,"num_bytes_recovered":5319,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6978},{"shards":"2,3","objects":3077}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":149374,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5378,"num_bytes_recovered":5395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724516864,"data_stored":757606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051592,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182410,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107836,"kb_used_data":26732,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94264004,"statfs":{"total":96636764160,"available":96526340096,"internally_reserved":0,"allocated":27373568,"data_stored":101253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107764,"kb_used_data":26660,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94264076,"statfs":{"total":96636764160,"available":96526413824,"internally_reserved":0,"allocated":27299840,"data_stored":101235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27164672,"data_stored":6632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27164672,"data_stored":6632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:02.295 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:02.483 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:02.484 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:02.498 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":596,"stamp":"2026-02-22T20:52:01.116497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":149374,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5378,"num_bytes_recovered":5395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":223756,"kb_used_data":61548,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188519924,"statfs":{"total":193273528320,"available":193044402176,"internally_reserved":0,"allocated":63025152,"data_stored":204527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3416,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1708,"num_bytes_recovered":1708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001464"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":26071,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:57.280472+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:57.280472+0000","last_peered":"2026-02-22T20:51:57.280472+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:57.280472+0000","last_undegraded":"2026-02-22T20:51:57.280472+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6154,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5320,"num_bytes_recovered":5319,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6978},{"shards":"2,3","objects":3077}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":149374,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5378,"num_bytes_recovered":5395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":732848128,"data_stored":759640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051592,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182410,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111928,"kb_used_data":30824,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94259912,"statfs":{"total":96636764160,"available":96522149888,"internally_reserved":0,"allocated":31563776,"data_stored":102276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111828,"kb_used_data":30724,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94260012,"statfs":{"total":96636764160,"available":96522252288,"internally_reserved":0,"allocated":31461376,"data_stored":102251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31330304,"data_stored":7649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31330304,"data_stored":7649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:02.500 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:02.683 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:02.684 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:02.697 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":596,"stamp":"2026-02-22T20:52:01.116497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":149374,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5378,"num_bytes_recovered":5395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":223756,"kb_used_data":61548,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188519924,"statfs":{"total":193273528320,"available":193044402176,"internally_reserved":0,"allocated":63025152,"data_stored":204527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3416,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1708,"num_bytes_recovered":1708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001464"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":26071,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:51:57.280472+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:51:57.280472+0000","last_peered":"2026-02-22T20:51:57.280472+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:51:57.280472+0000","last_undegraded":"2026-02-22T20:51:57.280472+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6154,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5320,"num_bytes_recovered":5319,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":6978},{"shards":"2,3","objects":3077}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":149374,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5378,"num_bytes_recovered":5395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":732848128,"data_stored":759640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051592,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182410,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111928,"kb_used_data":30824,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94259912,"statfs":{"total":96636764160,"available":96522149888,"internally_reserved":0,"allocated":31563776,"data_stored":102276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111828,"kb_used_data":30724,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94260012,"statfs":{"total":96636764160,"available":96522252288,"internally_reserved":0,"allocated":31461376,"data_stored":102251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31330304,"data_stored":7649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31330304,"data_stored":7649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10252, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188123+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:51:26.188123+0000', 'last_peered': '2026-02-22T20:51:26.188123+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:51:26.188123+0000', 'last_undegraded': '2026-02-22T20:51:26.188123+0000', 'last_fullsized': '2026-02-22T20:51:26.188123+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10378, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187383+0000', 'last_change': '2026-02-22T20:51:26.187530+0000', 'last_active': '2026-02-22T20:51:26.187383+0000', 'last_peered': '2026-02-22T20:51:26.187383+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:51:26.187383+0000', 'last_undegraded': '2026-02-22T20:51:26.187383+0000', 'last_fullsized': '2026-02-22T20:51:26.187383+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20566, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10169, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187844+0000', 'last_change': '2026-02-22T20:51:26.187976+0000', 'last_active': '2026-02-22T20:51:26.187844+0000', 'last_peered': '2026-02-22T20:51:26.187844+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:51:26.187844+0000', 'last_undegraded': '2026-02-22T20:51:26.187844+0000', 'last_fullsized': '2026-02-22T20:51:26.187844+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20202, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10495, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190360+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:51:26.190360+0000', 'last_peered': '2026-02-22T20:51:26.190360+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:51:26.190360+0000', 'last_undegraded': '2026-02-22T20:51:26.190360+0000', 'last_fullsized': '2026-02-22T20:51:26.190360+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 26071, 'reported_epoch': 31, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-22T20:51:57.280472+0000', 'last_change': '2026-02-22T20:51:24.247835+0000', 'last_active': '2026-02-22T20:51:57.280472+0000', 'last_peered': '2026-02-22T20:51:57.280472+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:51:15.334499+0000', 'last_became_peered': '2026-02-22T20:51:15.334499+0000', 'last_unstale': '2026-02-22T20:51:57.280472+0000', 'last_undegraded': '2026-02-22T20:51:57.280472+0000', 'last_fullsized': '2026-02-22T20:51:15.268613+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 6154, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5320, 'num_bytes_recovered': 5319, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 6978}, {'shards': '2,3', 'objects': 3077}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10399, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188015+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:51:26.188015+0000', 'last_peered': '2026-02-22T20:51:26.188015+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:51:26.188015+0000', 'last_undegraded': '2026-02-22T20:51:26.188015+0000', 'last_fullsized': '2026-02-22T20:51:26.188015+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:02.698 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10301, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187898+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:51:26.187898+0000', 'last_peered': '2026-02-22T20:51:26.187898+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:51:26.187898+0000', 'last_undegraded': '2026-02-22T20:51:26.187898+0000', 'last_fullsized': '2026-02-22T20:51:26.187898+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:02.699 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:02.699 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10291, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190677+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:51:26.190677+0000', 'last_peered': '2026-02-22T20:51:26.190677+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:51:26.190677+0000', 'last_undegraded': '2026-02-22T20:51:26.190677+0000', 'last_fullsized': '2026-02-22T20:51:26.190677+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:02.699 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:02.921 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:02.921 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1139,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":31,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":229126144,"bytes_avail":193044402176,"bytes_total":193273528320,"misplaced_objects":149374,"misplaced_total":163508,"misplaced_ratio":0.91355774640996157,"recovering_objects_per_sec":142,"recovering_bytes_per_sec":142,"recovering_keys_per_sec":0,"num_objects_recovered":1708,"num_bytes_recovered":1708,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:52:02.937 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:02.937 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:03.125 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:03.126 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:03.141 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":597,"stamp":"2026-02-22T20:52:03.116745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":147086,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6522,"num_bytes_recovered":6539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":223756,"kb_used_data":61548,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188519924,"statfs":{"total":193273528320,"available":193044402176,"internally_reserved":0,"allocated":63025152,"data_stored":204527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5704,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2852,"num_bytes_recovered":2852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001469"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":29503,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:52:02.282729+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:52:02.282729+0000","last_peered":"2026-02-22T20:52:02.282729+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:52:02.282729+0000","last_undegraded":"2026-02-22T20:52:02.282729+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3866,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6464,"num_bytes_recovered":6463,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":8122},{"shards":"2,3","objects":1933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":147086,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6522,"num_bytes_recovered":6539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":732848128,"data_stored":759640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051592,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182411,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111928,"kb_used_data":30824,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94259912,"statfs":{"total":96636764160,"available":96522149888,"internally_reserved":0,"allocated":31563776,"data_stored":102276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111828,"kb_used_data":30724,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94260012,"statfs":{"total":96636764160,"available":96522252288,"internally_reserved":0,"allocated":31461376,"data_stored":102251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31330304,"data_stored":7649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31330304,"data_stored":7649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:06.143 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:06.328 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:06.329 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:06.342 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":598,"stamp":"2026-02-22T20:52:05.117015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":147086,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6522,"num_bytes_recovered":6539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":233460,"kb_used_data":71252,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188510220,"statfs":{"total":193273528320,"available":193034465280,"internally_reserved":0,"allocated":72962048,"data_stored":206953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-4130,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2065,"num_bytes_recovered":2065,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001487"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":29503,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:52:02.282729+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:52:02.282729+0000","last_peered":"2026-02-22T20:52:02.282729+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:52:02.282729+0000","last_undegraded":"2026-02-22T20:52:02.282729+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3866,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6464,"num_bytes_recovered":6463,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":8122},{"shards":"2,3","objects":1933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":147086,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6522,"num_bytes_recovered":6539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":744030208,"data_stored":762370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051593,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182411,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116128,"kb_used_data":35024,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94255712,"statfs":{"total":96636764160,"available":96517849088,"internally_reserved":0,"allocated":35864576,"data_stored":103326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117332,"kb_used_data":36228,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94254508,"statfs":{"total":96636764160,"available":96516616192,"internally_reserved":0,"allocated":37097472,"data_stored":103627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36921344,"data_stored":9014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36921344,"data_stored":9014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:06.344 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:06.528 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:06.529 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:06.541 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":598,"stamp":"2026-02-22T20:52:05.117015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":147086,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6522,"num_bytes_recovered":6539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":233460,"kb_used_data":71252,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188510220,"statfs":{"total":193273528320,"available":193034465280,"internally_reserved":0,"allocated":72962048,"data_stored":206953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-4130,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2065,"num_bytes_recovered":2065,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001487"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":29503,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:52:02.282729+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:52:02.282729+0000","last_peered":"2026-02-22T20:52:02.282729+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:52:02.282729+0000","last_undegraded":"2026-02-22T20:52:02.282729+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3866,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6464,"num_bytes_recovered":6463,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":8122},{"shards":"2,3","objects":1933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":147086,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6522,"num_bytes_recovered":6539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":744030208,"data_stored":762370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051593,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182411,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116128,"kb_used_data":35024,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94255712,"statfs":{"total":96636764160,"available":96517849088,"internally_reserved":0,"allocated":35864576,"data_stored":103326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117332,"kb_used_data":36228,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94254508,"statfs":{"total":96636764160,"available":96516616192,"internally_reserved":0,"allocated":37097472,"data_stored":103627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36921344,"data_stored":9014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36921344,"data_stored":9014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:06.543 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:06.543 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10252, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188123+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:51:26.188123+0000', 'last_peered': '2026-02-22T20:51:26.188123+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:51:26.188123+0000', 'last_undegraded': '2026-02-22T20:51:26.188123+0000', 'last_fullsized': '2026-02-22T20:51:26.188123+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:06.544 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:06.544 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10378, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187383+0000', 'last_change': '2026-02-22T20:51:26.187530+0000', 'last_active': '2026-02-22T20:51:26.187383+0000', 'last_peered': '2026-02-22T20:51:26.187383+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:51:26.187383+0000', 'last_undegraded': '2026-02-22T20:51:26.187383+0000', 'last_fullsized': '2026-02-22T20:51:26.187383+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20566, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:06.544 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:52:06.544 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10169, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187844+0000', 'last_change': '2026-02-22T20:51:26.187976+0000', 'last_active': '2026-02-22T20:51:26.187844+0000', 'last_peered': '2026-02-22T20:51:26.187844+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:51:26.187844+0000', 'last_undegraded': '2026-02-22T20:51:26.187844+0000', 'last_fullsized': '2026-02-22T20:51:26.187844+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20202, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:06.544 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:52:06.544 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10495, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190360+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:51:26.190360+0000', 'last_peered': '2026-02-22T20:51:26.190360+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:51:26.190360+0000', 'last_undegraded': '2026-02-22T20:51:26.190360+0000', 'last_fullsized': '2026-02-22T20:51:26.190360+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:06.544 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:52:06.545 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 29503, 'reported_epoch': 31, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-22T20:52:02.282729+0000', 'last_change': '2026-02-22T20:51:24.247835+0000', 'last_active': '2026-02-22T20:52:02.282729+0000', 'last_peered': '2026-02-22T20:52:02.282729+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:51:15.334499+0000', 'last_became_peered': '2026-02-22T20:51:15.334499+0000', 'last_unstale': '2026-02-22T20:52:02.282729+0000', 'last_undegraded': '2026-02-22T20:52:02.282729+0000', 'last_fullsized': '2026-02-22T20:51:15.268613+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3866, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6464, 'num_bytes_recovered': 6463, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 8122}, {'shards': '2,3', 'objects': 1933}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:06.545 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:52:06.545 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10399, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188015+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:51:26.188015+0000', 'last_peered': '2026-02-22T20:51:26.188015+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:51:26.188015+0000', 'last_undegraded': '2026-02-22T20:51:26.188015+0000', 'last_fullsized': '2026-02-22T20:51:26.188015+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:06.545 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:06.545 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10301, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187898+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:51:26.187898+0000', 'last_peered': '2026-02-22T20:51:26.187898+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:51:26.187898+0000', 'last_undegraded': '2026-02-22T20:51:26.187898+0000', 'last_fullsized': '2026-02-22T20:51:26.187898+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:06.545 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:06.546 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10291, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190677+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:51:26.190677+0000', 'last_peered': '2026-02-22T20:51:26.190677+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:51:26.190677+0000', 'last_undegraded': '2026-02-22T20:51:26.190677+0000', 'last_fullsized': '2026-02-22T20:51:26.190677+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:06.546 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:06.767 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:06.767 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1143,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":31,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":239063040,"bytes_avail":193034465280,"bytes_total":193273528320,"misplaced_objects":147086,"misplaced_total":163508,"misplaced_ratio":0.89956454730043789,"recovering_objects_per_sec":172,"recovering_bytes_per_sec":172,"recovering_keys_per_sec":0,"num_objects_recovered":2065,"num_bytes_recovered":2065,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:52:06.782 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:06.782 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:06.963 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:06.964 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:06.978 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":598,"stamp":"2026-02-22T20:52:05.117015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":147086,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6522,"num_bytes_recovered":6539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":233460,"kb_used_data":71252,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188510220,"statfs":{"total":193273528320,"available":193034465280,"internally_reserved":0,"allocated":72962048,"data_stored":206953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-4130,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2065,"num_bytes_recovered":2065,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001487"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":29503,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:52:02.282729+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:52:02.282729+0000","last_peered":"2026-02-22T20:52:02.282729+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:52:02.282729+0000","last_undegraded":"2026-02-22T20:52:02.282729+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3866,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6464,"num_bytes_recovered":6463,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":8122},{"shards":"2,3","objects":1933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":147086,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6522,"num_bytes_recovered":6539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":744030208,"data_stored":762370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051593,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182411,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116128,"kb_used_data":35024,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94255712,"statfs":{"total":96636764160,"available":96517849088,"internally_reserved":0,"allocated":35864576,"data_stored":103326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117332,"kb_used_data":36228,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94254508,"statfs":{"total":96636764160,"available":96516616192,"internally_reserved":0,"allocated":37097472,"data_stored":103627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36921344,"data_stored":9014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36921344,"data_stored":9014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:09.981 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:10.171 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:10.172 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:10.185 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":602,"stamp":"2026-02-22T20:52:09.134994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":143606,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8262,"num_bytes_recovered":8279,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":233460,"kb_used_data":71252,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188510220,"statfs":{"total":193273528320,"available":193034465280,"internally_reserved":0,"allocated":72962048,"data_stored":206953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5768,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2884,"num_bytes_recovered":2884,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.018496"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":34723,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:52:07.285201+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:52:07.285201+0000","last_peered":"2026-02-22T20:52:07.285201+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:52:07.285201+0000","last_undegraded":"2026-02-22T20:52:07.285201+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":386,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8204,"num_bytes_recovered":8203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":9862},{"shards":"2,3","objects":193}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":143606,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8262,"num_bytes_recovered":8279,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":744030208,"data_stored":762370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051594,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182412,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116128,"kb_used_data":35024,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94255712,"statfs":{"total":96636764160,"available":96517849088,"internally_reserved":0,"allocated":35864576,"data_stored":103326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117332,"kb_used_data":36228,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94254508,"statfs":{"total":96636764160,"available":96516616192,"internally_reserved":0,"allocated":37097472,"data_stored":103627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36921344,"data_stored":9014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36921344,"data_stored":9014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:10.187 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:10.377 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:10.377 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:10.391 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":602,"stamp":"2026-02-22T20:52:09.134994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":143606,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8262,"num_bytes_recovered":8279,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":233460,"kb_used_data":71252,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188510220,"statfs":{"total":193273528320,"available":193034465280,"internally_reserved":0,"allocated":72962048,"data_stored":206953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5768,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2884,"num_bytes_recovered":2884,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.018496"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":34723,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:52:07.285201+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:52:07.285201+0000","last_peered":"2026-02-22T20:52:07.285201+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:52:07.285201+0000","last_undegraded":"2026-02-22T20:52:07.285201+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":386,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8204,"num_bytes_recovered":8203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":9862},{"shards":"2,3","objects":193}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":143606,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8262,"num_bytes_recovered":8279,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":744030208,"data_stored":762370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051594,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182412,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116128,"kb_used_data":35024,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94255712,"statfs":{"total":96636764160,"available":96517849088,"internally_reserved":0,"allocated":35864576,"data_stored":103326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117332,"kb_used_data":36228,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94254508,"statfs":{"total":96636764160,"available":96516616192,"internally_reserved":0,"allocated":37097472,"data_stored":103627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36921344,"data_stored":9014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36921344,"data_stored":9014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:10.392 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:10.392 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10252, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188123+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:51:26.188123+0000', 'last_peered': '2026-02-22T20:51:26.188123+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:51:26.188123+0000', 'last_undegraded': '2026-02-22T20:51:26.188123+0000', 'last_fullsized': '2026-02-22T20:51:26.188123+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:10.393 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:10.393 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 10378, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187383+0000', 'last_change': '2026-02-22T20:51:26.187530+0000', 'last_active': '2026-02-22T20:51:26.187383+0000', 'last_peered': '2026-02-22T20:51:26.187383+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:51:26.187383+0000', 'last_undegraded': '2026-02-22T20:51:26.187383+0000', 'last_fullsized': '2026-02-22T20:51:26.187383+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20566, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:10.393 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:52:10.393 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 10169, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187844+0000', 'last_change': '2026-02-22T20:51:26.187976+0000', 'last_active': '2026-02-22T20:51:26.187844+0000', 'last_peered': '2026-02-22T20:51:26.187844+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:51:26.187844+0000', 'last_undegraded': '2026-02-22T20:51:26.187844+0000', 'last_fullsized': '2026-02-22T20:51:26.187844+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20202, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:10.393 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:52:10.393 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10495, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190360+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:51:26.190360+0000', 'last_peered': '2026-02-22T20:51:26.190360+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:51:26.190360+0000', 'last_undegraded': '2026-02-22T20:51:26.190360+0000', 'last_fullsized': '2026-02-22T20:51:26.190360+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:10.393 INFO:tasks.osd_recovery.ceph_manager:PG 1.3 is not active+clean 2026-02-22T20:52:10.394 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.3', 'version': "17'10055", 'reported_seq': 34723, 'reported_epoch': 31, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2026-02-22T20:52:07.285201+0000', 'last_change': '2026-02-22T20:51:24.247835+0000', 'last_active': '2026-02-22T20:52:07.285201+0000', 'last_peered': '2026-02-22T20:52:07.285201+0000', 'last_clean': '2026-02-22T20:50:30.642683+0000', 'last_became_active': '2026-02-22T20:51:15.334499+0000', 'last_became_peered': '2026-02-22T20:51:15.334499+0000', 'last_unstale': '2026-02-22T20:52:07.285201+0000', 'last_undegraded': '2026-02-22T20:52:07.285201+0000', 'last_fullsized': '2026-02-22T20:51:15.268613+0000', 'mapping_epoch': 26, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620, 'num_objects': 10055, 'num_object_clones': 0, 'num_object_copies': 20110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 386, 'num_objects_unfound': 0, 'num_objects_dirty': 10055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10054, 'num_write_kb': 10082, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8204, 'num_bytes_recovered': 8203, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2', 'objects': 9862}, {'shards': '2,3', 'objects': 193}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:10.394 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:52:10.394 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10399, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.188015+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:51:26.188015+0000', 'last_peered': '2026-02-22T20:51:26.188015+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:51:26.188015+0000', 'last_undegraded': '2026-02-22T20:51:26.188015+0000', 'last_fullsized': '2026-02-22T20:51:26.188015+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:10.394 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:10.394 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10301, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.187898+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:51:26.187898+0000', 'last_peered': '2026-02-22T20:51:26.187898+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:51:26.187898+0000', 'last_undegraded': '2026-02-22T20:51:26.187898+0000', 'last_fullsized': '2026-02-22T20:51:26.187898+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:10.394 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:10.394 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10291, 'reported_epoch': 31, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:51:26.190677+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:51:26.190677+0000', 'last_peered': '2026-02-22T20:51:26.190677+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:51:26.190677+0000', 'last_undegraded': '2026-02-22T20:51:26.190677+0000', 'last_fullsized': '2026-02-22T20:51:26.190677+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:10.395 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:10.626 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:10.626 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1147,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":33,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":7},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":239063040,"bytes_avail":193034465280,"bytes_total":193273528320,"misplaced_objects":143606,"misplaced_total":163508,"misplaced_ratio":0.87828118501846997,"recovering_objects_per_sec":288,"recovering_bytes_per_sec":288,"recovering_keys_per_sec":0,"num_objects_recovered":2884,"num_bytes_recovered":2884,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":5,"modified":"2026-02-22T20:49:11.095473+0000","services":{}},"progress_events":{}} 2026-02-22T20:52:10.640 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:10.640 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:10.830 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:10.831 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:10.846 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":602,"stamp":"2026-02-22T20:52:09.134994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":143606,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8262,"num_bytes_recovered":8279,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":233460,"kb_used_data":71252,"kb_used_omap":35,"kb_used_meta":162140,"kb_avail":188510220,"statfs":{"total":193273528320,"available":193034465280,"internally_reserved":0,"allocated":72962048,"data_stored":206953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":35954,"internal_metadata":166032270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-5768,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2884,"num_bytes_recovered":2884,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.018496"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10252,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188123+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:51:26.188123+0000","last_peered":"2026-02-22T20:51:26.188123+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:51:26.188123+0000","last_undegraded":"2026-02-22T20:51:26.188123+0000","last_fullsized":"2026-02-22T20:51:26.188123+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":10378,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187383+0000","last_change":"2026-02-22T20:51:26.187530+0000","last_active":"2026-02-22T20:51:26.187383+0000","last_peered":"2026-02-22T20:51:26.187383+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:51:26.187383+0000","last_undegraded":"2026-02-22T20:51:26.187383+0000","last_fullsized":"2026-02-22T20:51:26.187383+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20566,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":10169,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187844+0000","last_change":"2026-02-22T20:51:26.187976+0000","last_active":"2026-02-22T20:51:26.187844+0000","last_peered":"2026-02-22T20:51:26.187844+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:51:26.187844+0000","last_undegraded":"2026-02-22T20:51:26.187844+0000","last_fullsized":"2026-02-22T20:51:26.187844+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20202,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10495,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190360+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:51:26.190360+0000","last_peered":"2026-02-22T20:51:26.190360+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:51:26.190360+0000","last_undegraded":"2026-02-22T20:51:26.190360+0000","last_fullsized":"2026-02-22T20:51:26.190360+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":34723,"reported_epoch":31,"state":"active+recovering+undersized+remapped","last_fresh":"2026-02-22T20:52:07.285201+0000","last_change":"2026-02-22T20:51:24.247835+0000","last_active":"2026-02-22T20:52:07.285201+0000","last_peered":"2026-02-22T20:52:07.285201+0000","last_clean":"2026-02-22T20:50:30.642683+0000","last_became_active":"2026-02-22T20:51:15.334499+0000","last_became_peered":"2026-02-22T20:51:15.334499+0000","last_unstale":"2026-02-22T20:52:07.285201+0000","last_undegraded":"2026-02-22T20:52:07.285201+0000","last_fullsized":"2026-02-22T20:51:15.268613+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":386,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8204,"num_bytes_recovered":8203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2","objects":9862},{"shards":"2,3","objects":193}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10399,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.188015+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:51:26.188015+0000","last_peered":"2026-02-22T20:51:26.188015+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:51:26.188015+0000","last_undegraded":"2026-02-22T20:51:26.188015+0000","last_fullsized":"2026-02-22T20:51:26.188015+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10301,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.187898+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:51:26.187898+0000","last_peered":"2026-02-22T20:51:26.187898+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:51:26.187898+0000","last_undegraded":"2026-02-22T20:51:26.187898+0000","last_fullsized":"2026-02-22T20:51:26.187898+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10291,"reported_epoch":31,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:51:26.190677+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:51:26.190677+0000","last_peered":"2026-02-22T20:51:26.190677+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:51:26.190677+0000","last_undegraded":"2026-02-22T20:51:26.190677+0000","last_fullsized":"2026-02-22T20:51:26.190677+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":143606,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8262,"num_bytes_recovered":8279,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":744030208,"data_stored":762370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":15,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051594,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182412,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705897,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116128,"kb_used_data":35024,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94255712,"statfs":{"total":96636764160,"available":96517849088,"internally_reserved":0,"allocated":35864576,"data_stored":103326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18628,"internal_metadata":83015484},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117332,"kb_used_data":36228,"kb_used_omap":16,"kb_used_meta":81071,"kb_avail":94254508,"statfs":{"total":96636764160,"available":96516616192,"internally_reserved":0,"allocated":37097472,"data_stored":103627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17326,"internal_metadata":83016786},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36921344,"data_stored":9014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36921344,"data_stored":9014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:13.847 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:14.039 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:14.039 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:14.054 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":604,"stamp":"2026-02-22T20:52:13.117847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":133206,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13463,"num_bytes_recovered":74612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":260408,"kb_used_data":98200,"kb_used_omap":38,"kb_used_meta":162137,"kb_avail":188483272,"statfs":{"total":193273528320,"available":193006870528,"internally_reserved":0,"allocated":100556800,"data_stored":330865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39209,"internal_metadata":166029015},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-13880,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6941,"num_bytes_recovered":68073,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000831"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10256,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136315+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:09.136315+0000","last_peered":"2026-02-22T20:52:09.136315+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:09.136315+0000","last_undegraded":"2026-02-22T20:52:09.136315+0000","last_fullsized":"2026-02-22T20:52:09.136315+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15264,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.143318+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:09.143318+0000","last_peered":"2026-02-22T20:52:09.143318+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:09.143318+0000","last_undegraded":"2026-02-22T20:52:09.143318+0000","last_fullsized":"2026-02-22T20:52:09.143318+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":20321,"reported_epoch":33,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:12.285525+0000","last_change":"2026-02-22T20:52:09.143399+0000","last_active":"2026-02-22T20:52:12.285525+0000","last_peered":"2026-02-22T20:52:12.285525+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:52:12.285525+0000","last_undegraded":"2026-02-22T20:52:12.285525+0000","last_fullsized":"2026-02-22T20:52:12.285525+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13440,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":18665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10499,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136095+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:09.136095+0000","last_peered":"2026-02-22T20:52:09.136095+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:09.136095+0000","last_undegraded":"2026-02-22T20:52:09.136095+0000","last_fullsized":"2026-02-22T20:52:09.136095+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10106,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-22T20:52:09.142121+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:09.142121+0000","last_peered":"2026-02-22T20:52:09.142121+0000","last_clean":"2026-02-22T20:52:09.142121+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:09.142121+0000","last_undegraded":"2026-02-22T20:52:09.142121+0000","last_fullsized":"2026-02-22T20:52:09.142121+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10403,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136094+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:52:09.136094+0000","last_peered":"2026-02-22T20:52:09.136094+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:09.136094+0000","last_undegraded":"2026-02-22T20:52:09.136094+0000","last_fullsized":"2026-02-22T20:52:09.136094+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10305,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136012+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:52:09.136012+0000","last_peered":"2026-02-22T20:52:09.136012+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:09.136012+0000","last_undegraded":"2026-02-22T20:52:09.136012+0000","last_fullsized":"2026-02-22T20:52:09.136012+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10295,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136040+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:09.136040+0000","last_peered":"2026-02-22T20:52:09.136040+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:09.136040+0000","last_undegraded":"2026-02-22T20:52:09.136040+0000","last_fullsized":"2026-02-22T20:52:09.136040+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":133206,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13463,"num_bytes_recovered":74612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":773918720,"data_stored":891907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051594,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182413,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":128276,"kb_used_data":47172,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94243564,"statfs":{"total":96636764160,"available":96505409536,"internally_reserved":0,"allocated":48304128,"data_stored":155615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19279,"internal_metadata":83014833},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132132,"kb_used_data":51028,"kb_used_omap":19,"kb_used_meta":81068,"kb_avail":94239708,"statfs":{"total":96636764160,"available":96501460992,"internally_reserved":0,"allocated":52252672,"data_stored":175250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19930,"internal_metadata":83014182},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":52109312,"data_stored":73842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":52109312,"data_stored":73842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334606336,"data_stored":372052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:14.056 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:14.248 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:14.249 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:14.263 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":604,"stamp":"2026-02-22T20:52:13.117847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":133206,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13463,"num_bytes_recovered":74612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":260408,"kb_used_data":98200,"kb_used_omap":38,"kb_used_meta":162137,"kb_avail":188483272,"statfs":{"total":193273528320,"available":193006870528,"internally_reserved":0,"allocated":100556800,"data_stored":330865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39209,"internal_metadata":166029015},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-13880,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6941,"num_bytes_recovered":68073,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000831"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10256,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136315+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:09.136315+0000","last_peered":"2026-02-22T20:52:09.136315+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:09.136315+0000","last_undegraded":"2026-02-22T20:52:09.136315+0000","last_fullsized":"2026-02-22T20:52:09.136315+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15264,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.143318+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:09.143318+0000","last_peered":"2026-02-22T20:52:09.143318+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:09.143318+0000","last_undegraded":"2026-02-22T20:52:09.143318+0000","last_fullsized":"2026-02-22T20:52:09.143318+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":20321,"reported_epoch":33,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:12.285525+0000","last_change":"2026-02-22T20:52:09.143399+0000","last_active":"2026-02-22T20:52:12.285525+0000","last_peered":"2026-02-22T20:52:12.285525+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:52:12.285525+0000","last_undegraded":"2026-02-22T20:52:12.285525+0000","last_fullsized":"2026-02-22T20:52:12.285525+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13440,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":18665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10499,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136095+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:09.136095+0000","last_peered":"2026-02-22T20:52:09.136095+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:09.136095+0000","last_undegraded":"2026-02-22T20:52:09.136095+0000","last_fullsized":"2026-02-22T20:52:09.136095+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10106,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-22T20:52:09.142121+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:09.142121+0000","last_peered":"2026-02-22T20:52:09.142121+0000","last_clean":"2026-02-22T20:52:09.142121+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:09.142121+0000","last_undegraded":"2026-02-22T20:52:09.142121+0000","last_fullsized":"2026-02-22T20:52:09.142121+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10403,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136094+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:52:09.136094+0000","last_peered":"2026-02-22T20:52:09.136094+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:09.136094+0000","last_undegraded":"2026-02-22T20:52:09.136094+0000","last_fullsized":"2026-02-22T20:52:09.136094+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10305,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136012+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:52:09.136012+0000","last_peered":"2026-02-22T20:52:09.136012+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:09.136012+0000","last_undegraded":"2026-02-22T20:52:09.136012+0000","last_fullsized":"2026-02-22T20:52:09.136012+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10295,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136040+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:09.136040+0000","last_peered":"2026-02-22T20:52:09.136040+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:09.136040+0000","last_undegraded":"2026-02-22T20:52:09.136040+0000","last_fullsized":"2026-02-22T20:52:09.136040+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":133206,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13463,"num_bytes_recovered":74612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":773918720,"data_stored":891907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051594,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182413,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":128276,"kb_used_data":47172,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94243564,"statfs":{"total":96636764160,"available":96505409536,"internally_reserved":0,"allocated":48304128,"data_stored":155615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19279,"internal_metadata":83014833},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132132,"kb_used_data":51028,"kb_used_omap":19,"kb_used_meta":81068,"kb_avail":94239708,"statfs":{"total":96636764160,"available":96501460992,"internally_reserved":0,"allocated":52252672,"data_stored":175250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19930,"internal_metadata":83014182},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":52109312,"data_stored":73842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":52109312,"data_stored":73842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334606336,"data_stored":372052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:14.264 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:14.264 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10256, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136315+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:52:09.136315+0000', 'last_peered': '2026-02-22T20:52:09.136315+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:09.136315+0000', 'last_undegraded': '2026-02-22T20:52:09.136315+0000', 'last_fullsized': '2026-02-22T20:52:09.136315+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:14.264 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:14.264 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 15264, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.143318+0000', 'last_change': '2026-02-22T20:52:09.142836+0000', 'last_active': '2026-02-22T20:52:09.143318+0000', 'last_peered': '2026-02-22T20:52:09.143318+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:52:09.143318+0000', 'last_undegraded': '2026-02-22T20:52:09.143318+0000', 'last_fullsized': '2026-02-22T20:52:09.143318+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17314, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1626, 'num_bytes_recovered': 16909, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:14.264 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:52:14.264 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 20321, 'reported_epoch': 33, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:52:12.285525+0000', 'last_change': '2026-02-22T20:52:09.143399+0000', 'last_active': '2026-02-22T20:52:12.285525+0000', 'last_peered': '2026-02-22T20:52:12.285525+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:52:12.285525+0000', 'last_undegraded': '2026-02-22T20:52:12.285525+0000', 'last_fullsized': '2026-02-22T20:52:12.285525+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 13440, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3382, 'num_bytes_recovered': 18665, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:14.265 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:52:14.265 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10499, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136095+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:52:09.136095+0000', 'last_peered': '2026-02-22T20:52:09.136095+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:52:09.136095+0000', 'last_undegraded': '2026-02-22T20:52:09.136095+0000', 'last_fullsized': '2026-02-22T20:52:09.136095+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:14.265 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:52:14.265 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10403, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136094+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:52:09.136094+0000', 'last_peered': '2026-02-22T20:52:09.136094+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:52:09.136094+0000', 'last_undegraded': '2026-02-22T20:52:09.136094+0000', 'last_fullsized': '2026-02-22T20:52:09.136094+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:14.265 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:14.265 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10305, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136012+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:52:09.136012+0000', 'last_peered': '2026-02-22T20:52:09.136012+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:09.136012+0000', 'last_undegraded': '2026-02-22T20:52:09.136012+0000', 'last_fullsized': '2026-02-22T20:52:09.136012+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:14.265 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:14.265 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10295, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136040+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:52:09.136040+0000', 'last_peered': '2026-02-22T20:52:09.136040+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:09.136040+0000', 'last_undegraded': '2026-02-22T20:52:09.136040+0000', 'last_fullsized': '2026-02-22T20:52:09.136040+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:14.265 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:14.495 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:14.495 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1151,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":33,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":6},{"state_name":"active+clean","count":1},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":266657792,"bytes_avail":193006870528,"bytes_total":193273528320,"misplaced_objects":133206,"misplaced_total":163508,"misplaced_ratio":0.81467573452063524,"recovering_objects_per_sec":867,"recovering_bytes_per_sec":8508,"recovering_keys_per_sec":0,"num_objects_recovered":6941,"num_bytes_recovered":68073,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":6,"modified":"2026-02-22T20:52:13.117888+0000","services":{"osd":{"daemons":{"summary":"","3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2026-02-22T20:52:14.510 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:14.510 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:14.700 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:14.701 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:14.715 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":604,"stamp":"2026-02-22T20:52:13.117847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":133206,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13463,"num_bytes_recovered":74612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":260408,"kb_used_data":98200,"kb_used_omap":38,"kb_used_meta":162137,"kb_avail":188483272,"statfs":{"total":193273528320,"available":193006870528,"internally_reserved":0,"allocated":100556800,"data_stored":330865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39209,"internal_metadata":166029015},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-13880,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6941,"num_bytes_recovered":68073,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000831"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10256,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136315+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:09.136315+0000","last_peered":"2026-02-22T20:52:09.136315+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:09.136315+0000","last_undegraded":"2026-02-22T20:52:09.136315+0000","last_fullsized":"2026-02-22T20:52:09.136315+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15264,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.143318+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:09.143318+0000","last_peered":"2026-02-22T20:52:09.143318+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:09.143318+0000","last_undegraded":"2026-02-22T20:52:09.143318+0000","last_fullsized":"2026-02-22T20:52:09.143318+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":20321,"reported_epoch":33,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:12.285525+0000","last_change":"2026-02-22T20:52:09.143399+0000","last_active":"2026-02-22T20:52:12.285525+0000","last_peered":"2026-02-22T20:52:12.285525+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:52:12.285525+0000","last_undegraded":"2026-02-22T20:52:12.285525+0000","last_fullsized":"2026-02-22T20:52:12.285525+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13440,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":18665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10499,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136095+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:09.136095+0000","last_peered":"2026-02-22T20:52:09.136095+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:09.136095+0000","last_undegraded":"2026-02-22T20:52:09.136095+0000","last_fullsized":"2026-02-22T20:52:09.136095+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10106,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-22T20:52:09.142121+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:09.142121+0000","last_peered":"2026-02-22T20:52:09.142121+0000","last_clean":"2026-02-22T20:52:09.142121+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:09.142121+0000","last_undegraded":"2026-02-22T20:52:09.142121+0000","last_fullsized":"2026-02-22T20:52:09.142121+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10403,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136094+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:52:09.136094+0000","last_peered":"2026-02-22T20:52:09.136094+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:09.136094+0000","last_undegraded":"2026-02-22T20:52:09.136094+0000","last_fullsized":"2026-02-22T20:52:09.136094+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10305,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136012+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:52:09.136012+0000","last_peered":"2026-02-22T20:52:09.136012+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:09.136012+0000","last_undegraded":"2026-02-22T20:52:09.136012+0000","last_fullsized":"2026-02-22T20:52:09.136012+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10295,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136040+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:09.136040+0000","last_peered":"2026-02-22T20:52:09.136040+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:09.136040+0000","last_undegraded":"2026-02-22T20:52:09.136040+0000","last_fullsized":"2026-02-22T20:52:09.136040+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":133206,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13463,"num_bytes_recovered":74612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":773918720,"data_stored":891907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051594,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182413,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705898,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":128276,"kb_used_data":47172,"kb_used_omap":18,"kb_used_meta":81069,"kb_avail":94243564,"statfs":{"total":96636764160,"available":96505409536,"internally_reserved":0,"allocated":48304128,"data_stored":155615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19279,"internal_metadata":83014833},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132132,"kb_used_data":51028,"kb_used_omap":19,"kb_used_meta":81068,"kb_avail":94239708,"statfs":{"total":96636764160,"available":96501460992,"internally_reserved":0,"allocated":52252672,"data_stored":175250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19930,"internal_metadata":83014182},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":52109312,"data_stored":73842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":52109312,"data_stored":73842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334606336,"data_stored":372052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:17.718 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:17.909 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:17.910 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:17.923 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":606,"stamp":"2026-02-22T20:52:17.118325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":133206,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13463,"num_bytes_recovered":74612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":372172,"kb_used_data":137772,"kb_used_omap":38,"kb_used_meta":234329,"kb_avail":188371508,"statfs":{"total":193273528320,"available":192892424192,"internally_reserved":0,"allocated":141078528,"data_stored":359429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39860,"internal_metadata":239952972},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10400,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5201,"num_bytes_recovered":66333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.984113"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10256,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136315+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:09.136315+0000","last_peered":"2026-02-22T20:52:09.136315+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:09.136315+0000","last_undegraded":"2026-02-22T20:52:09.136315+0000","last_fullsized":"2026-02-22T20:52:09.136315+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15264,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.143318+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:09.143318+0000","last_peered":"2026-02-22T20:52:09.143318+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:09.143318+0000","last_undegraded":"2026-02-22T20:52:09.143318+0000","last_fullsized":"2026-02-22T20:52:09.143318+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":20321,"reported_epoch":33,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:12.285525+0000","last_change":"2026-02-22T20:52:09.143399+0000","last_active":"2026-02-22T20:52:12.285525+0000","last_peered":"2026-02-22T20:52:12.285525+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:52:12.285525+0000","last_undegraded":"2026-02-22T20:52:12.285525+0000","last_fullsized":"2026-02-22T20:52:12.285525+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13440,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":18665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10499,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136095+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:09.136095+0000","last_peered":"2026-02-22T20:52:09.136095+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:09.136095+0000","last_undegraded":"2026-02-22T20:52:09.136095+0000","last_fullsized":"2026-02-22T20:52:09.136095+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10106,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-22T20:52:09.142121+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:09.142121+0000","last_peered":"2026-02-22T20:52:09.142121+0000","last_clean":"2026-02-22T20:52:09.142121+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:09.142121+0000","last_undegraded":"2026-02-22T20:52:09.142121+0000","last_fullsized":"2026-02-22T20:52:09.142121+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10403,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136094+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:52:09.136094+0000","last_peered":"2026-02-22T20:52:09.136094+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:09.136094+0000","last_undegraded":"2026-02-22T20:52:09.136094+0000","last_fullsized":"2026-02-22T20:52:09.136094+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10305,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136012+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:52:09.136012+0000","last_peered":"2026-02-22T20:52:09.136012+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:09.136012+0000","last_undegraded":"2026-02-22T20:52:09.136012+0000","last_fullsized":"2026-02-22T20:52:09.136012+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10295,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136040+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:09.136040+0000","last_peered":"2026-02-22T20:52:09.136040+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:09.136040+0000","last_undegraded":"2026-02-22T20:52:09.136040+0000","last_fullsized":"2026-02-22T20:52:09.136040+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":133206,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13463,"num_bytes_recovered":74612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":777146368,"data_stored":892695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051595,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182413,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":185984,"kb_used_data":68784,"kb_used_omap":19,"kb_used_meta":117164,"kb_avail":94185856,"statfs":{"total":96636764160,"available":96446316544,"internally_reserved":0,"allocated":70434816,"data_stored":179689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19930,"internal_metadata":119976486},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":186188,"kb_used_data":68988,"kb_used_omap":19,"kb_used_meta":117164,"kb_avail":94185652,"statfs":{"total":96636764160,"available":96446107648,"internally_reserved":0,"allocated":70643712,"data_stored":179740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19930,"internal_metadata":119976486},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":74326016,"data_stored":94546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":74326016,"data_stored":94546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334606336,"data_stored":372052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":293888000,"data_stored":331551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:17.924 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:18.116 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:18.116 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:18.131 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":606,"stamp":"2026-02-22T20:52:17.118325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":133206,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13463,"num_bytes_recovered":74612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":372172,"kb_used_data":137772,"kb_used_omap":38,"kb_used_meta":234329,"kb_avail":188371508,"statfs":{"total":193273528320,"available":192892424192,"internally_reserved":0,"allocated":141078528,"data_stored":359429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39860,"internal_metadata":239952972},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10400,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5201,"num_bytes_recovered":66333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.984113"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10256,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136315+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:09.136315+0000","last_peered":"2026-02-22T20:52:09.136315+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:09.136315+0000","last_undegraded":"2026-02-22T20:52:09.136315+0000","last_fullsized":"2026-02-22T20:52:09.136315+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15264,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.143318+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:09.143318+0000","last_peered":"2026-02-22T20:52:09.143318+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:09.143318+0000","last_undegraded":"2026-02-22T20:52:09.143318+0000","last_fullsized":"2026-02-22T20:52:09.143318+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":20321,"reported_epoch":33,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:12.285525+0000","last_change":"2026-02-22T20:52:09.143399+0000","last_active":"2026-02-22T20:52:12.285525+0000","last_peered":"2026-02-22T20:52:12.285525+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:52:12.285525+0000","last_undegraded":"2026-02-22T20:52:12.285525+0000","last_fullsized":"2026-02-22T20:52:12.285525+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13440,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":18665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10499,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136095+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:09.136095+0000","last_peered":"2026-02-22T20:52:09.136095+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:09.136095+0000","last_undegraded":"2026-02-22T20:52:09.136095+0000","last_fullsized":"2026-02-22T20:52:09.136095+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10106,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-22T20:52:09.142121+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:09.142121+0000","last_peered":"2026-02-22T20:52:09.142121+0000","last_clean":"2026-02-22T20:52:09.142121+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:09.142121+0000","last_undegraded":"2026-02-22T20:52:09.142121+0000","last_fullsized":"2026-02-22T20:52:09.142121+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10403,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136094+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:52:09.136094+0000","last_peered":"2026-02-22T20:52:09.136094+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:09.136094+0000","last_undegraded":"2026-02-22T20:52:09.136094+0000","last_fullsized":"2026-02-22T20:52:09.136094+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10305,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136012+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:52:09.136012+0000","last_peered":"2026-02-22T20:52:09.136012+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:09.136012+0000","last_undegraded":"2026-02-22T20:52:09.136012+0000","last_fullsized":"2026-02-22T20:52:09.136012+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10295,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136040+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:09.136040+0000","last_peered":"2026-02-22T20:52:09.136040+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:09.136040+0000","last_undegraded":"2026-02-22T20:52:09.136040+0000","last_fullsized":"2026-02-22T20:52:09.136040+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":133206,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13463,"num_bytes_recovered":74612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":777146368,"data_stored":892695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051595,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182413,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":185984,"kb_used_data":68784,"kb_used_omap":19,"kb_used_meta":117164,"kb_avail":94185856,"statfs":{"total":96636764160,"available":96446316544,"internally_reserved":0,"allocated":70434816,"data_stored":179689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19930,"internal_metadata":119976486},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":186188,"kb_used_data":68988,"kb_used_omap":19,"kb_used_meta":117164,"kb_avail":94185652,"statfs":{"total":96636764160,"available":96446107648,"internally_reserved":0,"allocated":70643712,"data_stored":179740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19930,"internal_metadata":119976486},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":74326016,"data_stored":94546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":74326016,"data_stored":94546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334606336,"data_stored":372052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":293888000,"data_stored":331551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:18.131 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:18.131 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10256, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136315+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:52:09.136315+0000', 'last_peered': '2026-02-22T20:52:09.136315+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:09.136315+0000', 'last_undegraded': '2026-02-22T20:52:09.136315+0000', 'last_fullsized': '2026-02-22T20:52:09.136315+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:18.132 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:18.132 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 15264, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.143318+0000', 'last_change': '2026-02-22T20:52:09.142836+0000', 'last_active': '2026-02-22T20:52:09.143318+0000', 'last_peered': '2026-02-22T20:52:09.143318+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:52:09.143318+0000', 'last_undegraded': '2026-02-22T20:52:09.143318+0000', 'last_fullsized': '2026-02-22T20:52:09.143318+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17314, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1626, 'num_bytes_recovered': 16909, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:18.132 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:52:18.132 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 20321, 'reported_epoch': 33, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:52:12.285525+0000', 'last_change': '2026-02-22T20:52:09.143399+0000', 'last_active': '2026-02-22T20:52:12.285525+0000', 'last_peered': '2026-02-22T20:52:12.285525+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:52:12.285525+0000', 'last_undegraded': '2026-02-22T20:52:12.285525+0000', 'last_fullsized': '2026-02-22T20:52:12.285525+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 11, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 13440, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3382, 'num_bytes_recovered': 18665, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10101}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:18.132 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:52:18.132 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10499, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136095+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:52:09.136095+0000', 'last_peered': '2026-02-22T20:52:09.136095+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:52:09.136095+0000', 'last_undegraded': '2026-02-22T20:52:09.136095+0000', 'last_fullsized': '2026-02-22T20:52:09.136095+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:18.132 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:52:18.132 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10403, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136094+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:52:09.136094+0000', 'last_peered': '2026-02-22T20:52:09.136094+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:52:09.136094+0000', 'last_undegraded': '2026-02-22T20:52:09.136094+0000', 'last_fullsized': '2026-02-22T20:52:09.136094+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:18.132 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:18.132 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10305, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136012+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:52:09.136012+0000', 'last_peered': '2026-02-22T20:52:09.136012+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:09.136012+0000', 'last_undegraded': '2026-02-22T20:52:09.136012+0000', 'last_fullsized': '2026-02-22T20:52:09.136012+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:18.132 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:18.132 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10295, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136040+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:52:09.136040+0000', 'last_peered': '2026-02-22T20:52:09.136040+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:09.136040+0000', 'last_undegraded': '2026-02-22T20:52:09.136040+0000', 'last_fullsized': '2026-02-22T20:52:09.136040+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:18.132 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:18.362 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:18.362 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1155,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":33,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":6},{"state_name":"active+remapped+backfilling","count":1},{"state_name":"active+clean","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":381104128,"bytes_avail":192892424192,"bytes_total":193273528320,"misplaced_objects":133206,"misplaced_total":163508,"misplaced_ratio":0.81467573452063524,"recovering_objects_per_sec":578,"recovering_bytes_per_sec":7383,"recovering_keys_per_sec":0,"num_objects_recovered":5201,"num_bytes_recovered":66333,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":6,"modified":"2026-02-22T20:52:13.117888+0000","services":{"osd":{"daemons":{"summary":"","3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2026-02-22T20:52:18.376 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:18.376 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:18.567 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:18.567 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:18.583 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":606,"stamp":"2026-02-22T20:52:17.118325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":133206,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13463,"num_bytes_recovered":74612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":372172,"kb_used_data":137772,"kb_used_omap":38,"kb_used_meta":234329,"kb_avail":188371508,"statfs":{"total":193273528320,"available":192892424192,"internally_reserved":0,"allocated":141078528,"data_stored":359429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39860,"internal_metadata":239952972},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10400,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5201,"num_bytes_recovered":66333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.984113"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10256,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136315+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:09.136315+0000","last_peered":"2026-02-22T20:52:09.136315+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:09.136315+0000","last_undegraded":"2026-02-22T20:52:09.136315+0000","last_fullsized":"2026-02-22T20:52:09.136315+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15264,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.143318+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:09.143318+0000","last_peered":"2026-02-22T20:52:09.143318+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:09.143318+0000","last_undegraded":"2026-02-22T20:52:09.143318+0000","last_fullsized":"2026-02-22T20:52:09.143318+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":20321,"reported_epoch":33,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:12.285525+0000","last_change":"2026-02-22T20:52:09.143399+0000","last_active":"2026-02-22T20:52:12.285525+0000","last_peered":"2026-02-22T20:52:12.285525+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:52:12.285525+0000","last_undegraded":"2026-02-22T20:52:12.285525+0000","last_fullsized":"2026-02-22T20:52:12.285525+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13440,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":18665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10101}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10499,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136095+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:09.136095+0000","last_peered":"2026-02-22T20:52:09.136095+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:09.136095+0000","last_undegraded":"2026-02-22T20:52:09.136095+0000","last_fullsized":"2026-02-22T20:52:09.136095+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10106,"reported_epoch":33,"state":"active+clean","last_fresh":"2026-02-22T20:52:09.142121+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:09.142121+0000","last_peered":"2026-02-22T20:52:09.142121+0000","last_clean":"2026-02-22T20:52:09.142121+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:09.142121+0000","last_undegraded":"2026-02-22T20:52:09.142121+0000","last_fullsized":"2026-02-22T20:52:09.142121+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10403,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136094+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:52:09.136094+0000","last_peered":"2026-02-22T20:52:09.136094+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:09.136094+0000","last_undegraded":"2026-02-22T20:52:09.136094+0000","last_fullsized":"2026-02-22T20:52:09.136094+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10305,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136012+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:52:09.136012+0000","last_peered":"2026-02-22T20:52:09.136012+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:09.136012+0000","last_undegraded":"2026-02-22T20:52:09.136012+0000","last_fullsized":"2026-02-22T20:52:09.136012+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10295,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136040+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:09.136040+0000","last_peered":"2026-02-22T20:52:09.136040+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:09.136040+0000","last_undegraded":"2026-02-22T20:52:09.136040+0000","last_fullsized":"2026-02-22T20:52:09.136040+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":133206,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13463,"num_bytes_recovered":74612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":777146368,"data_stored":892695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80255,"ondisk_log_size":80255,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051595,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182413,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":185984,"kb_used_data":68784,"kb_used_omap":19,"kb_used_meta":117164,"kb_avail":94185856,"statfs":{"total":96636764160,"available":96446316544,"internally_reserved":0,"allocated":70434816,"data_stored":179689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19930,"internal_metadata":119976486},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":186188,"kb_used_data":68988,"kb_used_omap":19,"kb_used_meta":117164,"kb_avail":94185652,"statfs":{"total":96636764160,"available":96446107648,"internally_reserved":0,"allocated":70643712,"data_stored":179740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19930,"internal_metadata":119976486},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":74326016,"data_stored":94546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":74326016,"data_stored":94546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334606336,"data_stored":372052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":293888000,"data_stored":331551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:21.585 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:21.776 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:21.776 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:21.791 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":610,"stamp":"2026-02-22T20:52:21.118764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":119766,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20182,"num_bytes_recovered":96637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":491720,"kb_used_data":185128,"kb_used_omap":39,"kb_used_meta":306520,"kb_avail":188251960,"statfs":{"total":193273528320,"available":192770007040,"internally_reserved":0,"allocated":189571072,"data_stored":405257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40511,"internal_metadata":313876929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-13440,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6719,"num_bytes_recovered":22025,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000917"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10256,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136315+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:09.136315+0000","last_peered":"2026-02-22T20:52:09.136315+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:09.136315+0000","last_undegraded":"2026-02-22T20:52:09.136315+0000","last_fullsized":"2026-02-22T20:52:09.136315+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15264,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.143318+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:09.143318+0000","last_peered":"2026-02-22T20:52:09.143318+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:09.143318+0000","last_undegraded":"2026-02-22T20:52:09.143318+0000","last_fullsized":"2026-02-22T20:52:09.143318+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40480,"reported_epoch":34,"state":"peering","last_fresh":"2026-02-22T20:52:19.152205+0000","last_change":"2026-02-22T20:52:19.152205+0000","last_active":"2026-02-22T20:52:19.152205+0000","last_peered":"2026-02-22T20:52:18.449442+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:52:19.152205+0000","last_undegraded":"2026-02-22T20:52:19.152205+0000","last_fullsized":"2026-02-22T20:52:19.152205+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10499,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136095+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:09.136095+0000","last_peered":"2026-02-22T20:52:09.136095+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:09.136095+0000","last_undegraded":"2026-02-22T20:52:09.136095+0000","last_fullsized":"2026-02-22T20:52:09.136095+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10108,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-22T20:52:19.151468+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:19.151468+0000","last_peered":"2026-02-22T20:52:19.151468+0000","last_clean":"2026-02-22T20:52:19.151468+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:19.151468+0000","last_undegraded":"2026-02-22T20:52:19.151468+0000","last_fullsized":"2026-02-22T20:52:19.151468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10403,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136094+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:52:09.136094+0000","last_peered":"2026-02-22T20:52:09.136094+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:09.136094+0000","last_undegraded":"2026-02-22T20:52:09.136094+0000","last_fullsized":"2026-02-22T20:52:09.136094+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10305,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136012+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:52:09.136012+0000","last_peered":"2026-02-22T20:52:09.136012+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:09.136012+0000","last_undegraded":"2026-02-22T20:52:09.136012+0000","last_fullsized":"2026-02-22T20:52:09.136012+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10295,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136040+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:09.136040+0000","last_peered":"2026-02-22T20:52:09.136040+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:09.136040+0000","last_undegraded":"2026-02-22T20:52:09.136040+0000","last_fullsized":"2026-02-22T20:52:09.136040+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":119766,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20182,"num_bytes_recovered":96637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":821293056,"data_stored":903519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051596,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182414,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":243828,"kb_used_data":90532,"kb_used_omap":19,"kb_used_meta":153260,"kb_avail":94128012,"statfs":{"total":96636764160,"available":96387084288,"internally_reserved":0,"allocated":92704768,"data_stored":200429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19930,"internal_metadata":156938790},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247892,"kb_used_data":94596,"kb_used_omap":20,"kb_used_meta":153259,"kb_avail":94123948,"statfs":{"total":96636764160,"available":96382922752,"internally_reserved":0,"allocated":96866304,"data_stored":204828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20581,"internal_metadata":156938139},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":96706560,"data_stored":100033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96706560,"data_stored":100033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333991936,"data_stored":371902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":293888000,"data_stored":331551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:21.793 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:21.981 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:21.981 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:21.996 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":610,"stamp":"2026-02-22T20:52:21.118764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":119766,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20182,"num_bytes_recovered":96637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":491720,"kb_used_data":185128,"kb_used_omap":39,"kb_used_meta":306520,"kb_avail":188251960,"statfs":{"total":193273528320,"available":192770007040,"internally_reserved":0,"allocated":189571072,"data_stored":405257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40511,"internal_metadata":313876929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-13440,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6719,"num_bytes_recovered":22025,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000917"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10256,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136315+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:09.136315+0000","last_peered":"2026-02-22T20:52:09.136315+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:09.136315+0000","last_undegraded":"2026-02-22T20:52:09.136315+0000","last_fullsized":"2026-02-22T20:52:09.136315+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15264,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.143318+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:09.143318+0000","last_peered":"2026-02-22T20:52:09.143318+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:09.143318+0000","last_undegraded":"2026-02-22T20:52:09.143318+0000","last_fullsized":"2026-02-22T20:52:09.143318+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40480,"reported_epoch":34,"state":"peering","last_fresh":"2026-02-22T20:52:19.152205+0000","last_change":"2026-02-22T20:52:19.152205+0000","last_active":"2026-02-22T20:52:19.152205+0000","last_peered":"2026-02-22T20:52:18.449442+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:52:19.152205+0000","last_undegraded":"2026-02-22T20:52:19.152205+0000","last_fullsized":"2026-02-22T20:52:19.152205+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10499,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136095+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:09.136095+0000","last_peered":"2026-02-22T20:52:09.136095+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:09.136095+0000","last_undegraded":"2026-02-22T20:52:09.136095+0000","last_fullsized":"2026-02-22T20:52:09.136095+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10108,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-22T20:52:19.151468+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:19.151468+0000","last_peered":"2026-02-22T20:52:19.151468+0000","last_clean":"2026-02-22T20:52:19.151468+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:19.151468+0000","last_undegraded":"2026-02-22T20:52:19.151468+0000","last_fullsized":"2026-02-22T20:52:19.151468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10403,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136094+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:52:09.136094+0000","last_peered":"2026-02-22T20:52:09.136094+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:09.136094+0000","last_undegraded":"2026-02-22T20:52:09.136094+0000","last_fullsized":"2026-02-22T20:52:09.136094+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10305,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136012+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:52:09.136012+0000","last_peered":"2026-02-22T20:52:09.136012+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:09.136012+0000","last_undegraded":"2026-02-22T20:52:09.136012+0000","last_fullsized":"2026-02-22T20:52:09.136012+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10295,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136040+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:09.136040+0000","last_peered":"2026-02-22T20:52:09.136040+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:09.136040+0000","last_undegraded":"2026-02-22T20:52:09.136040+0000","last_fullsized":"2026-02-22T20:52:09.136040+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":119766,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20182,"num_bytes_recovered":96637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":821293056,"data_stored":903519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051596,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182414,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":243828,"kb_used_data":90532,"kb_used_omap":19,"kb_used_meta":153260,"kb_avail":94128012,"statfs":{"total":96636764160,"available":96387084288,"internally_reserved":0,"allocated":92704768,"data_stored":200429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19930,"internal_metadata":156938790},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247892,"kb_used_data":94596,"kb_used_omap":20,"kb_used_meta":153259,"kb_avail":94123948,"statfs":{"total":96636764160,"available":96382922752,"internally_reserved":0,"allocated":96866304,"data_stored":204828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20581,"internal_metadata":156938139},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":96706560,"data_stored":100033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96706560,"data_stored":100033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333991936,"data_stored":371902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":293888000,"data_stored":331551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:21.997 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:21.997 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10256, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136315+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:52:09.136315+0000', 'last_peered': '2026-02-22T20:52:09.136315+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:09.136315+0000', 'last_undegraded': '2026-02-22T20:52:09.136315+0000', 'last_fullsized': '2026-02-22T20:52:09.136315+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:21.997 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:21.998 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 15264, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.143318+0000', 'last_change': '2026-02-22T20:52:09.142836+0000', 'last_active': '2026-02-22T20:52:09.143318+0000', 'last_peered': '2026-02-22T20:52:09.143318+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:52:09.143318+0000', 'last_undegraded': '2026-02-22T20:52:09.143318+0000', 'last_fullsized': '2026-02-22T20:52:09.143318+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17314, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1626, 'num_bytes_recovered': 16909, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:21.998 INFO:tasks.osd_recovery.ceph_manager:PG 1.5 is not active+clean 2026-02-22T20:52:21.998 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.5', 'version': "17'10101", 'reported_seq': 40480, 'reported_epoch': 34, 'state': 'peering', 'last_fresh': '2026-02-22T20:52:19.152205+0000', 'last_change': '2026-02-22T20:52:19.152205+0000', 'last_active': '2026-02-22T20:52:19.152205+0000', 'last_peered': '2026-02-22T20:52:18.449442+0000', 'last_clean': '2026-02-22T20:50:32.195860+0000', 'last_became_active': '2026-02-22T20:51:26.182459+0000', 'last_became_peered': '2026-02-22T20:51:26.182459+0000', 'last_unstale': '2026-02-22T20:52:19.152205+0000', 'last_undegraded': '2026-02-22T20:52:19.152205+0000', 'last_fullsized': '2026-02-22T20:52:19.152205+0000', 'mapping_epoch': 34, 'log_start': "17'101", 'ondisk_log_start': "17'101", 'created': 10, 'last_epoch_clean': 33, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10000, 'log_dups_size': 0, 'ondisk_log_size': 10000, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000206202, 'stat_sum': {'num_bytes': 40690, 'num_objects': 10101, 'num_object_clones': 0, 'num_object_copies': 20202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 10129, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10101, 'num_bytes_recovered': 40690, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2026-02-22T20:52:21.998 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:52:21.998 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10499, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136095+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:52:09.136095+0000', 'last_peered': '2026-02-22T20:52:09.136095+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:52:09.136095+0000', 'last_undegraded': '2026-02-22T20:52:09.136095+0000', 'last_fullsized': '2026-02-22T20:52:09.136095+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:21.998 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:52:21.998 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 10403, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136094+0000', 'last_change': '2026-02-22T20:51:26.188166+0000', 'last_active': '2026-02-22T20:52:09.136094+0000', 'last_peered': '2026-02-22T20:52:09.136094+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:52:09.136094+0000', 'last_undegraded': '2026-02-22T20:52:09.136094+0000', 'last_fullsized': '2026-02-22T20:52:09.136094+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20224, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 76, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:21.999 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:21.999 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 10305, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136012+0000', 'last_change': '2026-02-22T20:51:26.188067+0000', 'last_active': '2026-02-22T20:52:09.136012+0000', 'last_peered': '2026-02-22T20:52:09.136012+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:09.136012+0000', 'last_undegraded': '2026-02-22T20:52:09.136012+0000', 'last_fullsized': '2026-02-22T20:52:09.136012+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20476, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:21.999 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:21.999 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10295, 'reported_epoch': 33, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:09.136040+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:52:09.136040+0000', 'last_peered': '2026-02-22T20:52:09.136040+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:09.136040+0000', 'last_undegraded': '2026-02-22T20:52:09.136040+0000', 'last_fullsized': '2026-02-22T20:52:09.136040+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:21.999 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:22.231 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:22.231 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1158,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":35,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":6},{"state_name":"active+clean","count":1},{"state_name":"peering","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":503521280,"bytes_avail":192770007040,"bytes_total":193273528320,"inactive_pgs_ratio":0.125,"misplaced_objects":119766,"misplaced_total":163508,"misplaced_ratio":0.7324779215695868,"recovering_objects_per_sec":839,"recovering_bytes_per_sec":2752,"recovering_keys_per_sec":0,"num_objects_recovered":6719,"num_bytes_recovered":22025,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":6,"modified":"2026-02-22T20:52:13.117888+0000","services":{"osd":{"daemons":{"summary":"","3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2026-02-22T20:52:22.246 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:22.246 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:22.437 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:22.437 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:22.451 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":610,"stamp":"2026-02-22T20:52:21.118764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":119766,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20182,"num_bytes_recovered":96637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":491720,"kb_used_data":185128,"kb_used_omap":39,"kb_used_meta":306520,"kb_avail":188251960,"statfs":{"total":193273528320,"available":192770007040,"internally_reserved":0,"allocated":189571072,"data_stored":405257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40511,"internal_metadata":313876929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-13440,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6719,"num_bytes_recovered":22025,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000917"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10256,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136315+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:09.136315+0000","last_peered":"2026-02-22T20:52:09.136315+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:09.136315+0000","last_undegraded":"2026-02-22T20:52:09.136315+0000","last_fullsized":"2026-02-22T20:52:09.136315+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15264,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.143318+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:09.143318+0000","last_peered":"2026-02-22T20:52:09.143318+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:09.143318+0000","last_undegraded":"2026-02-22T20:52:09.143318+0000","last_fullsized":"2026-02-22T20:52:09.143318+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40480,"reported_epoch":34,"state":"peering","last_fresh":"2026-02-22T20:52:19.152205+0000","last_change":"2026-02-22T20:52:19.152205+0000","last_active":"2026-02-22T20:52:19.152205+0000","last_peered":"2026-02-22T20:52:18.449442+0000","last_clean":"2026-02-22T20:50:32.195860+0000","last_became_active":"2026-02-22T20:51:26.182459+0000","last_became_peered":"2026-02-22T20:51:26.182459+0000","last_unstale":"2026-02-22T20:52:19.152205+0000","last_undegraded":"2026-02-22T20:52:19.152205+0000","last_fullsized":"2026-02-22T20:52:19.152205+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10499,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136095+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:09.136095+0000","last_peered":"2026-02-22T20:52:09.136095+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:09.136095+0000","last_undegraded":"2026-02-22T20:52:09.136095+0000","last_fullsized":"2026-02-22T20:52:09.136095+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10108,"reported_epoch":34,"state":"active+clean","last_fresh":"2026-02-22T20:52:19.151468+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:19.151468+0000","last_peered":"2026-02-22T20:52:19.151468+0000","last_clean":"2026-02-22T20:52:19.151468+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:19.151468+0000","last_undegraded":"2026-02-22T20:52:19.151468+0000","last_fullsized":"2026-02-22T20:52:19.151468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":10403,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136094+0000","last_change":"2026-02-22T20:51:26.188166+0000","last_active":"2026-02-22T20:52:09.136094+0000","last_peered":"2026-02-22T20:52:09.136094+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:09.136094+0000","last_undegraded":"2026-02-22T20:52:09.136094+0000","last_fullsized":"2026-02-22T20:52:09.136094+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20224,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":10305,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136012+0000","last_change":"2026-02-22T20:51:26.188067+0000","last_active":"2026-02-22T20:52:09.136012+0000","last_peered":"2026-02-22T20:52:09.136012+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:09.136012+0000","last_undegraded":"2026-02-22T20:52:09.136012+0000","last_fullsized":"2026-02-22T20:52:09.136012+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20476,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10295,"reported_epoch":33,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:09.136040+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:09.136040+0000","last_peered":"2026-02-22T20:52:09.136040+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:09.136040+0000","last_undegraded":"2026-02-22T20:52:09.136040+0000","last_fullsized":"2026-02-22T20:52:09.136040+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":119766,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20182,"num_bytes_recovered":96637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":821293056,"data_stored":903519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051596,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182414,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705900,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":243828,"kb_used_data":90532,"kb_used_omap":19,"kb_used_meta":153260,"kb_avail":94128012,"statfs":{"total":96636764160,"available":96387084288,"internally_reserved":0,"allocated":92704768,"data_stored":200429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19930,"internal_metadata":156938790},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247892,"kb_used_data":94596,"kb_used_omap":20,"kb_used_meta":153259,"kb_avail":94123948,"statfs":{"total":96636764160,"available":96382922752,"internally_reserved":0,"allocated":96866304,"data_stored":204828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20581,"internal_metadata":156938139},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":96706560,"data_stored":100033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96706560,"data_stored":100033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333991936,"data_stored":371902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":293888000,"data_stored":331551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:25.453 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:25.643 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:25.643 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:25.657 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":612,"stamp":"2026-02-22T20:52:25.119210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":111442,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24345,"num_bytes_recovered":116083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":535128,"kb_used_data":228472,"kb_used_omap":41,"kb_used_meta":306582,"kb_avail":188208552,"statfs":{"total":193273528320,"available":192725557248,"internally_reserved":0,"allocated":233955328,"data_stored":456760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42464,"internal_metadata":313940512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21764,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10882,"num_bytes_recovered":41471,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000884"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10260,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154919+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:20.154919+0000","last_peered":"2026-02-22T20:52:20.154919+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:20.154919+0000","last_undegraded":"2026-02-22T20:52:20.154919+0000","last_fullsized":"2026-02-22T20:52:20.154919+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15268,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154447+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:20.154447+0000","last_peered":"2026-02-22T20:52:20.154447+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:20.154447+0000","last_undegraded":"2026-02-22T20:52:20.154447+0000","last_fullsized":"2026-02-22T20:52:20.154447+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40489,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.159970+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:20.159970+0000","last_peered":"2026-02-22T20:52:20.159970+0000","last_clean":"2026-02-22T20:52:20.159970+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:20.159970+0000","last_undegraded":"2026-02-22T20:52:20.159970+0000","last_fullsized":"2026-02-22T20:52:20.159970+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10503,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154945+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:20.154945+0000","last_peered":"2026-02-22T20:52:20.154945+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:20.154945+0000","last_undegraded":"2026-02-22T20:52:20.154945+0000","last_fullsized":"2026-02-22T20:52:20.154945+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10110,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.154759+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:20.154759+0000","last_peered":"2026-02-22T20:52:20.154759+0000","last_clean":"2026-02-22T20:52:20.154759+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:20.154759+0000","last_undegraded":"2026-02-22T20:52:20.154759+0000","last_fullsized":"2026-02-22T20:52:20.154759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":17249,"reported_epoch":35,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:22.285800+0000","last_change":"2026-02-22T20:52:20.160858+0000","last_active":"2026-02-22T20:52:22.285800+0000","last_peered":"2026-02-22T20:52:22.285800+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:22.285800+0000","last_undegraded":"2026-02-22T20:52:22.285800+0000","last_fullsized":"2026-02-22T20:52:22.285800+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":15664,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2338,"num_bytes_recovered":17639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15962,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.160722+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:20.160722+0000","last_peered":"2026-02-22T20:52:20.160722+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:20.160722+0000","last_undegraded":"2026-02-22T20:52:20.160722+0000","last_fullsized":"2026-02-22T20:52:20.160722+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10299,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154899+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:20.154899+0000","last_peered":"2026-02-22T20:52:20.154899+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:20.154899+0000","last_undegraded":"2026-02-22T20:52:20.154899+0000","last_fullsized":"2026-02-22T20:52:20.154899+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":111442,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24345,"num_bytes_recovered":116083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":823619584,"data_stored":904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051597,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182415,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265380,"kb_used_data":112084,"kb_used_omap":20,"kb_used_meta":153259,"kb_avail":94106460,"statfs":{"total":96636764160,"available":96365015040,"internally_reserved":0,"allocated":114774016,"data_stored":227842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":156937488},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269748,"kb_used_data":116388,"kb_used_omap":20,"kb_used_meta":153323,"kb_avail":94102092,"statfs":{"total":96636764160,"available":96360542208,"internally_reserved":0,"allocated":119181312,"data_stored":228918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":157003024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":119058432,"data_stored":120770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":119058432,"data_stored":120770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333012992,"data_stored":371663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":252489728,"data_stored":290861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:25.658 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:25.847 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:25.847 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:25.862 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":612,"stamp":"2026-02-22T20:52:25.119210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":111442,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24345,"num_bytes_recovered":116083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":535128,"kb_used_data":228472,"kb_used_omap":41,"kb_used_meta":306582,"kb_avail":188208552,"statfs":{"total":193273528320,"available":192725557248,"internally_reserved":0,"allocated":233955328,"data_stored":456760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42464,"internal_metadata":313940512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21764,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10882,"num_bytes_recovered":41471,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000884"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10260,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154919+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:20.154919+0000","last_peered":"2026-02-22T20:52:20.154919+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:20.154919+0000","last_undegraded":"2026-02-22T20:52:20.154919+0000","last_fullsized":"2026-02-22T20:52:20.154919+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15268,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154447+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:20.154447+0000","last_peered":"2026-02-22T20:52:20.154447+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:20.154447+0000","last_undegraded":"2026-02-22T20:52:20.154447+0000","last_fullsized":"2026-02-22T20:52:20.154447+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40489,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.159970+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:20.159970+0000","last_peered":"2026-02-22T20:52:20.159970+0000","last_clean":"2026-02-22T20:52:20.159970+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:20.159970+0000","last_undegraded":"2026-02-22T20:52:20.159970+0000","last_fullsized":"2026-02-22T20:52:20.159970+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10503,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154945+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:20.154945+0000","last_peered":"2026-02-22T20:52:20.154945+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:20.154945+0000","last_undegraded":"2026-02-22T20:52:20.154945+0000","last_fullsized":"2026-02-22T20:52:20.154945+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10110,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.154759+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:20.154759+0000","last_peered":"2026-02-22T20:52:20.154759+0000","last_clean":"2026-02-22T20:52:20.154759+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:20.154759+0000","last_undegraded":"2026-02-22T20:52:20.154759+0000","last_fullsized":"2026-02-22T20:52:20.154759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":17249,"reported_epoch":35,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:22.285800+0000","last_change":"2026-02-22T20:52:20.160858+0000","last_active":"2026-02-22T20:52:22.285800+0000","last_peered":"2026-02-22T20:52:22.285800+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:22.285800+0000","last_undegraded":"2026-02-22T20:52:22.285800+0000","last_fullsized":"2026-02-22T20:52:22.285800+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":15664,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2338,"num_bytes_recovered":17639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15962,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.160722+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:20.160722+0000","last_peered":"2026-02-22T20:52:20.160722+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:20.160722+0000","last_undegraded":"2026-02-22T20:52:20.160722+0000","last_fullsized":"2026-02-22T20:52:20.160722+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10299,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154899+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:20.154899+0000","last_peered":"2026-02-22T20:52:20.154899+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:20.154899+0000","last_undegraded":"2026-02-22T20:52:20.154899+0000","last_fullsized":"2026-02-22T20:52:20.154899+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":111442,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24345,"num_bytes_recovered":116083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":823619584,"data_stored":904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051597,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182415,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265380,"kb_used_data":112084,"kb_used_omap":20,"kb_used_meta":153259,"kb_avail":94106460,"statfs":{"total":96636764160,"available":96365015040,"internally_reserved":0,"allocated":114774016,"data_stored":227842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":156937488},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269748,"kb_used_data":116388,"kb_used_omap":20,"kb_used_meta":153323,"kb_avail":94102092,"statfs":{"total":96636764160,"available":96360542208,"internally_reserved":0,"allocated":119181312,"data_stored":228918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":157003024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":119058432,"data_stored":120770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":119058432,"data_stored":120770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333012992,"data_stored":371663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":252489728,"data_stored":290861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:25.864 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:25.864 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10260, 'reported_epoch': 35, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:20.154919+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:52:20.154919+0000', 'last_peered': '2026-02-22T20:52:20.154919+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:20.154919+0000', 'last_undegraded': '2026-02-22T20:52:20.154919+0000', 'last_fullsized': '2026-02-22T20:52:20.154919+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:25.864 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:25.864 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 15268, 'reported_epoch': 35, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:20.154447+0000', 'last_change': '2026-02-22T20:52:09.142836+0000', 'last_active': '2026-02-22T20:52:20.154447+0000', 'last_peered': '2026-02-22T20:52:20.154447+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:52:20.154447+0000', 'last_undegraded': '2026-02-22T20:52:20.154447+0000', 'last_fullsized': '2026-02-22T20:52:20.154447+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17314, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1626, 'num_bytes_recovered': 16909, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:25.864 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:52:25.865 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10503, 'reported_epoch': 35, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:20.154945+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:52:20.154945+0000', 'last_peered': '2026-02-22T20:52:20.154945+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:52:20.154945+0000', 'last_undegraded': '2026-02-22T20:52:20.154945+0000', 'last_fullsized': '2026-02-22T20:52:20.154945+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:25.865 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:52:25.865 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 17249, 'reported_epoch': 35, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:52:22.285800+0000', 'last_change': '2026-02-22T20:52:20.160858+0000', 'last_active': '2026-02-22T20:52:22.285800+0000', 'last_peered': '2026-02-22T20:52:22.285800+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:52:22.285800+0000', 'last_undegraded': '2026-02-22T20:52:22.285800+0000', 'last_fullsized': '2026-02-22T20:52:22.285800+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 15664, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2338, 'num_bytes_recovered': 17639, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:25.865 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:25.865 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 15962, 'reported_epoch': 35, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:20.160722+0000', 'last_change': '2026-02-22T20:52:20.160334+0000', 'last_active': '2026-02-22T20:52:20.160722+0000', 'last_peered': '2026-02-22T20:52:20.160722+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:20.160722+0000', 'last_undegraded': '2026-02-22T20:52:20.160722+0000', 'last_fullsized': '2026-02-22T20:52:20.160722+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16712, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1883, 'num_bytes_recovered': 1883, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:25.865 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:25.865 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10299, 'reported_epoch': 35, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:20.154899+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:52:20.154899+0000', 'last_peered': '2026-02-22T20:52:20.154899+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:20.154899+0000', 'last_undegraded': '2026-02-22T20:52:20.154899+0000', 'last_fullsized': '2026-02-22T20:52:20.154899+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:25.866 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:26.099 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:26.099 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1162,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":35,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":5},{"state_name":"active+clean","count":2},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":547971072,"bytes_avail":192725557248,"bytes_total":193273528320,"misplaced_objects":111442,"misplaced_total":163508,"misplaced_ratio":0.68156909753651196,"recovering_objects_per_sec":1360,"recovering_bytes_per_sec":5183,"recovering_keys_per_sec":0,"num_objects_recovered":10882,"num_bytes_recovered":41471,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:52:26.112 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:26.112 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:26.304 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:26.305 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:26.318 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":612,"stamp":"2026-02-22T20:52:25.119210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":111442,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24345,"num_bytes_recovered":116083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":535128,"kb_used_data":228472,"kb_used_omap":41,"kb_used_meta":306582,"kb_avail":188208552,"statfs":{"total":193273528320,"available":192725557248,"internally_reserved":0,"allocated":233955328,"data_stored":456760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42464,"internal_metadata":313940512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21764,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10882,"num_bytes_recovered":41471,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000884"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10260,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154919+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:20.154919+0000","last_peered":"2026-02-22T20:52:20.154919+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:20.154919+0000","last_undegraded":"2026-02-22T20:52:20.154919+0000","last_fullsized":"2026-02-22T20:52:20.154919+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15268,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154447+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:20.154447+0000","last_peered":"2026-02-22T20:52:20.154447+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:20.154447+0000","last_undegraded":"2026-02-22T20:52:20.154447+0000","last_fullsized":"2026-02-22T20:52:20.154447+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40489,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.159970+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:20.159970+0000","last_peered":"2026-02-22T20:52:20.159970+0000","last_clean":"2026-02-22T20:52:20.159970+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:20.159970+0000","last_undegraded":"2026-02-22T20:52:20.159970+0000","last_fullsized":"2026-02-22T20:52:20.159970+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10503,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154945+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:20.154945+0000","last_peered":"2026-02-22T20:52:20.154945+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:20.154945+0000","last_undegraded":"2026-02-22T20:52:20.154945+0000","last_fullsized":"2026-02-22T20:52:20.154945+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10110,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.154759+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:20.154759+0000","last_peered":"2026-02-22T20:52:20.154759+0000","last_clean":"2026-02-22T20:52:20.154759+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:20.154759+0000","last_undegraded":"2026-02-22T20:52:20.154759+0000","last_fullsized":"2026-02-22T20:52:20.154759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":17249,"reported_epoch":35,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:22.285800+0000","last_change":"2026-02-22T20:52:20.160858+0000","last_active":"2026-02-22T20:52:22.285800+0000","last_peered":"2026-02-22T20:52:22.285800+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:22.285800+0000","last_undegraded":"2026-02-22T20:52:22.285800+0000","last_fullsized":"2026-02-22T20:52:22.285800+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":15664,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2338,"num_bytes_recovered":17639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15962,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.160722+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:20.160722+0000","last_peered":"2026-02-22T20:52:20.160722+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:20.160722+0000","last_undegraded":"2026-02-22T20:52:20.160722+0000","last_fullsized":"2026-02-22T20:52:20.160722+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10299,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154899+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:20.154899+0000","last_peered":"2026-02-22T20:52:20.154899+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:20.154899+0000","last_undegraded":"2026-02-22T20:52:20.154899+0000","last_fullsized":"2026-02-22T20:52:20.154899+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":111442,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24345,"num_bytes_recovered":116083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":823619584,"data_stored":904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051597,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182415,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265380,"kb_used_data":112084,"kb_used_omap":20,"kb_used_meta":153259,"kb_avail":94106460,"statfs":{"total":96636764160,"available":96365015040,"internally_reserved":0,"allocated":114774016,"data_stored":227842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":156937488},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269748,"kb_used_data":116388,"kb_used_omap":20,"kb_used_meta":153323,"kb_avail":94102092,"statfs":{"total":96636764160,"available":96360542208,"internally_reserved":0,"allocated":119181312,"data_stored":228918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":157003024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":119058432,"data_stored":120770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":119058432,"data_stored":120770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333012992,"data_stored":371663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":252489728,"data_stored":290861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:29.320 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:29.508 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:29.508 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:29.522 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":614,"stamp":"2026-02-22T20:52:29.119641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":100472,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29830,"num_bytes_recovered":121568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":535128,"kb_used_data":228472,"kb_used_omap":41,"kb_used_meta":306582,"kb_avail":188208552,"statfs":{"total":193273528320,"available":192725557248,"internally_reserved":0,"allocated":233955328,"data_stored":456760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42464,"internal_metadata":313940512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21832,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10916,"num_bytes_recovered":26199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.968983"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10260,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154919+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:20.154919+0000","last_peered":"2026-02-22T20:52:20.154919+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:20.154919+0000","last_undegraded":"2026-02-22T20:52:20.154919+0000","last_fullsized":"2026-02-22T20:52:20.154919+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15268,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154447+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:20.154447+0000","last_peered":"2026-02-22T20:52:20.154447+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:20.154447+0000","last_undegraded":"2026-02-22T20:52:20.154447+0000","last_fullsized":"2026-02-22T20:52:20.154447+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40489,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.159970+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:20.159970+0000","last_peered":"2026-02-22T20:52:20.159970+0000","last_clean":"2026-02-22T20:52:20.159970+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:20.159970+0000","last_undegraded":"2026-02-22T20:52:20.159970+0000","last_fullsized":"2026-02-22T20:52:20.159970+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10503,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154945+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:20.154945+0000","last_peered":"2026-02-22T20:52:20.154945+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:20.154945+0000","last_undegraded":"2026-02-22T20:52:20.154945+0000","last_fullsized":"2026-02-22T20:52:20.154945+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10110,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.154759+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:20.154759+0000","last_peered":"2026-02-22T20:52:20.154759+0000","last_clean":"2026-02-22T20:52:20.154759+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:20.154759+0000","last_undegraded":"2026-02-22T20:52:20.154759+0000","last_fullsized":"2026-02-22T20:52:20.154759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":33704,"reported_epoch":35,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:27.286558+0000","last_change":"2026-02-22T20:52:20.160858+0000","last_active":"2026-02-22T20:52:27.286558+0000","last_peered":"2026-02-22T20:52:27.286558+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:27.286558+0000","last_undegraded":"2026-02-22T20:52:27.286558+0000","last_fullsized":"2026-02-22T20:52:27.286558+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4694,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7823,"num_bytes_recovered":23124,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15962,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.160722+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:20.160722+0000","last_peered":"2026-02-22T20:52:20.160722+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:20.160722+0000","last_undegraded":"2026-02-22T20:52:20.160722+0000","last_fullsized":"2026-02-22T20:52:20.160722+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10299,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154899+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:20.154899+0000","last_peered":"2026-02-22T20:52:20.154899+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:20.154899+0000","last_undegraded":"2026-02-22T20:52:20.154899+0000","last_fullsized":"2026-02-22T20:52:20.154899+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":100472,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29830,"num_bytes_recovered":121568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":822390784,"data_stored":903764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051598,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182416,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265380,"kb_used_data":112084,"kb_used_omap":20,"kb_used_meta":153259,"kb_avail":94106460,"statfs":{"total":96636764160,"available":96365015040,"internally_reserved":0,"allocated":114774016,"data_stored":227842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":156937488},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269748,"kb_used_data":116388,"kb_used_omap":20,"kb_used_meta":153323,"kb_avail":94102092,"statfs":{"total":96636764160,"available":96360542208,"internally_reserved":0,"allocated":119181312,"data_stored":228918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":157003024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":119058432,"data_stored":120770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":119058432,"data_stored":120770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331784192,"data_stored":371363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":252489728,"data_stored":290861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:29.523 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:29.712 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:29.712 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:29.725 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":614,"stamp":"2026-02-22T20:52:29.119641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":100472,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29830,"num_bytes_recovered":121568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":535128,"kb_used_data":228472,"kb_used_omap":41,"kb_used_meta":306582,"kb_avail":188208552,"statfs":{"total":193273528320,"available":192725557248,"internally_reserved":0,"allocated":233955328,"data_stored":456760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42464,"internal_metadata":313940512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21832,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10916,"num_bytes_recovered":26199,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.968983"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10260,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154919+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:20.154919+0000","last_peered":"2026-02-22T20:52:20.154919+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:20.154919+0000","last_undegraded":"2026-02-22T20:52:20.154919+0000","last_fullsized":"2026-02-22T20:52:20.154919+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15268,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154447+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:20.154447+0000","last_peered":"2026-02-22T20:52:20.154447+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:20.154447+0000","last_undegraded":"2026-02-22T20:52:20.154447+0000","last_fullsized":"2026-02-22T20:52:20.154447+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40489,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.159970+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:20.159970+0000","last_peered":"2026-02-22T20:52:20.159970+0000","last_clean":"2026-02-22T20:52:20.159970+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:20.159970+0000","last_undegraded":"2026-02-22T20:52:20.159970+0000","last_fullsized":"2026-02-22T20:52:20.159970+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10503,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154945+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:20.154945+0000","last_peered":"2026-02-22T20:52:20.154945+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:20.154945+0000","last_undegraded":"2026-02-22T20:52:20.154945+0000","last_fullsized":"2026-02-22T20:52:20.154945+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10110,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.154759+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:20.154759+0000","last_peered":"2026-02-22T20:52:20.154759+0000","last_clean":"2026-02-22T20:52:20.154759+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:20.154759+0000","last_undegraded":"2026-02-22T20:52:20.154759+0000","last_fullsized":"2026-02-22T20:52:20.154759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":33704,"reported_epoch":35,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:27.286558+0000","last_change":"2026-02-22T20:52:20.160858+0000","last_active":"2026-02-22T20:52:27.286558+0000","last_peered":"2026-02-22T20:52:27.286558+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:27.286558+0000","last_undegraded":"2026-02-22T20:52:27.286558+0000","last_fullsized":"2026-02-22T20:52:27.286558+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4694,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7823,"num_bytes_recovered":23124,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15962,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.160722+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:20.160722+0000","last_peered":"2026-02-22T20:52:20.160722+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:20.160722+0000","last_undegraded":"2026-02-22T20:52:20.160722+0000","last_fullsized":"2026-02-22T20:52:20.160722+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10299,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154899+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:20.154899+0000","last_peered":"2026-02-22T20:52:20.154899+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:20.154899+0000","last_undegraded":"2026-02-22T20:52:20.154899+0000","last_fullsized":"2026-02-22T20:52:20.154899+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":100472,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29830,"num_bytes_recovered":121568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":822390784,"data_stored":903764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051598,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182416,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705901,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265380,"kb_used_data":112084,"kb_used_omap":20,"kb_used_meta":153259,"kb_avail":94106460,"statfs":{"total":96636764160,"available":96365015040,"internally_reserved":0,"allocated":114774016,"data_stored":227842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":156937488},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269748,"kb_used_data":116388,"kb_used_omap":20,"kb_used_meta":153323,"kb_avail":94102092,"statfs":{"total":96636764160,"available":96360542208,"internally_reserved":0,"allocated":119181312,"data_stored":228918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":157003024},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":119058432,"data_stored":120770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":119058432,"data_stored":120770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331784192,"data_stored":371363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":252489728,"data_stored":290861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:29.726 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:29.726 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 10260, 'reported_epoch': 35, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:20.154919+0000', 'last_change': '2026-02-22T20:51:26.188281+0000', 'last_active': '2026-02-22T20:52:20.154919+0000', 'last_peered': '2026-02-22T20:52:20.154919+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:20.154919+0000', 'last_undegraded': '2026-02-22T20:52:20.154919+0000', 'last_fullsized': '2026-02-22T20:52:20.154919+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20410, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:29.726 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:29.726 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 15268, 'reported_epoch': 35, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:20.154447+0000', 'last_change': '2026-02-22T20:52:09.142836+0000', 'last_active': '2026-02-22T20:52:20.154447+0000', 'last_peered': '2026-02-22T20:52:20.154447+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:52:20.154447+0000', 'last_undegraded': '2026-02-22T20:52:20.154447+0000', 'last_fullsized': '2026-02-22T20:52:20.154447+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17314, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1626, 'num_bytes_recovered': 16909, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:29.726 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:52:29.726 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 10503, 'reported_epoch': 35, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:20.154945+0000', 'last_change': '2026-02-22T20:51:26.190455+0000', 'last_active': '2026-02-22T20:52:20.154945+0000', 'last_peered': '2026-02-22T20:52:20.154945+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:52:20.154945+0000', 'last_undegraded': '2026-02-22T20:52:20.154945+0000', 'last_fullsized': '2026-02-22T20:52:20.154945+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20864, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:29.726 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:52:29.727 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 33704, 'reported_epoch': 35, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:52:27.286558+0000', 'last_change': '2026-02-22T20:52:20.160858+0000', 'last_active': '2026-02-22T20:52:27.286558+0000', 'last_peered': '2026-02-22T20:52:27.286558+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:52:27.286558+0000', 'last_undegraded': '2026-02-22T20:52:27.286558+0000', 'last_fullsized': '2026-02-22T20:52:27.286558+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4694, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7823, 'num_bytes_recovered': 23124, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:29.727 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:29.727 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 15962, 'reported_epoch': 35, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:20.160722+0000', 'last_change': '2026-02-22T20:52:20.160334+0000', 'last_active': '2026-02-22T20:52:20.160722+0000', 'last_peered': '2026-02-22T20:52:20.160722+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:20.160722+0000', 'last_undegraded': '2026-02-22T20:52:20.160722+0000', 'last_fullsized': '2026-02-22T20:52:20.160722+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16712, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1883, 'num_bytes_recovered': 1883, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:29.727 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:29.727 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10299, 'reported_epoch': 35, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:20.154899+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:52:20.154899+0000', 'last_peered': '2026-02-22T20:52:20.154899+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:20.154899+0000', 'last_undegraded': '2026-02-22T20:52:20.154899+0000', 'last_fullsized': '2026-02-22T20:52:20.154899+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:29.727 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:29.959 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:29.959 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1166,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":35,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":5},{"state_name":"active+clean","count":2},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":547971072,"bytes_avail":192725557248,"bytes_total":193273528320,"misplaced_objects":100472,"misplaced_total":163508,"misplaced_ratio":0.61447757907869949,"recovering_objects_per_sec":1094,"recovering_bytes_per_sec":2628,"recovering_keys_per_sec":0,"num_objects_recovered":10916,"num_bytes_recovered":26199,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:52:29.974 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:29.974 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:30.206 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:30.207 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:30.221 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":615,"stamp":"2026-02-22T20:52:30.169711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":100472,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29830,"num_bytes_recovered":121568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":651176,"kb_used_data":272392,"kb_used_omap":41,"kb_used_meta":378710,"kb_avail":188092504,"statfs":{"total":193273528320,"available":192606724096,"internally_reserved":0,"allocated":278929408,"data_stored":498336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42464,"internal_metadata":387799584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-19294,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9648,"num_bytes_recovered":24931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.018206"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":10260,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154919+0000","last_change":"2026-02-22T20:51:26.188281+0000","last_active":"2026-02-22T20:52:20.154919+0000","last_peered":"2026-02-22T20:52:20.154919+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:20.154919+0000","last_undegraded":"2026-02-22T20:52:20.154919+0000","last_fullsized":"2026-02-22T20:52:20.154919+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20410,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15268,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154447+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:20.154447+0000","last_peered":"2026-02-22T20:52:20.154447+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:20.154447+0000","last_undegraded":"2026-02-22T20:52:20.154447+0000","last_fullsized":"2026-02-22T20:52:20.154447+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40489,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.159970+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:20.159970+0000","last_peered":"2026-02-22T20:52:20.159970+0000","last_clean":"2026-02-22T20:52:20.159970+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:20.159970+0000","last_undegraded":"2026-02-22T20:52:20.159970+0000","last_fullsized":"2026-02-22T20:52:20.159970+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":10503,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154945+0000","last_change":"2026-02-22T20:51:26.190455+0000","last_active":"2026-02-22T20:52:20.154945+0000","last_peered":"2026-02-22T20:52:20.154945+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:20.154945+0000","last_undegraded":"2026-02-22T20:52:20.154945+0000","last_fullsized":"2026-02-22T20:52:20.154945+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20864,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10110,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.154759+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:20.154759+0000","last_peered":"2026-02-22T20:52:20.154759+0000","last_clean":"2026-02-22T20:52:20.154759+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:20.154759+0000","last_undegraded":"2026-02-22T20:52:20.154759+0000","last_fullsized":"2026-02-22T20:52:20.154759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":33704,"reported_epoch":35,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:27.286558+0000","last_change":"2026-02-22T20:52:20.160858+0000","last_active":"2026-02-22T20:52:27.286558+0000","last_peered":"2026-02-22T20:52:27.286558+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:27.286558+0000","last_undegraded":"2026-02-22T20:52:27.286558+0000","last_fullsized":"2026-02-22T20:52:27.286558+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4694,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7823,"num_bytes_recovered":23124,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15962,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.160722+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:20.160722+0000","last_peered":"2026-02-22T20:52:20.160722+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:20.160722+0000","last_undegraded":"2026-02-22T20:52:20.160722+0000","last_fullsized":"2026-02-22T20:52:20.160722+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10299,"reported_epoch":35,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:20.154899+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:20.154899+0000","last_peered":"2026-02-22T20:52:20.154899+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:20.154899+0000","last_undegraded":"2026-02-22T20:52:20.154899+0000","last_fullsized":"2026-02-22T20:52:20.154899+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":100472,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29830,"num_bytes_recovered":121568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":867012608,"data_stored":945254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051598,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182416,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323736,"kb_used_data":134344,"kb_used_omap":20,"kb_used_meta":189355,"kb_avail":94048104,"statfs":{"total":96636764160,"available":96305258496,"internally_reserved":0,"allocated":137568256,"data_stored":248705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":193899792},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327440,"kb_used_data":138048,"kb_used_omap":20,"kb_used_meta":189355,"kb_avail":94044400,"statfs":{"total":96636764160,"available":96301465600,"internally_reserved":0,"allocated":141361152,"data_stored":249631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":193899792},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":141369344,"data_stored":141515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":141369344,"data_stored":141515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331784192,"data_stored":371363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":252489728,"data_stored":290861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:33.223 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:33.411 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:33.411 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:33.424 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":618,"stamp":"2026-02-22T20:52:33.120112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":94476,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32830,"num_bytes_recovered":124568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":651176,"kb_used_data":272392,"kb_used_omap":41,"kb_used_meta":378710,"kb_avail":188092504,"statfs":{"total":193273528320,"available":192606724096,"internally_reserved":0,"allocated":278929408,"data_stored":498336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42464,"internal_metadata":387799584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-16966,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8485,"num_bytes_recovered":8485,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000902"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15698,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.181207+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:31.181207+0000","last_peered":"2026-02-22T20:52:31.181207+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:31.181207+0000","last_undegraded":"2026-02-22T20:52:31.181207+0000","last_fullsized":"2026-02-22T20:52:31.181207+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15272,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175902+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:31.175902+0000","last_peered":"2026-02-22T20:52:31.175902+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:31.175902+0000","last_undegraded":"2026-02-22T20:52:31.175902+0000","last_fullsized":"2026-02-22T20:52:31.175902+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40489,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.159970+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:20.159970+0000","last_peered":"2026-02-22T20:52:20.159970+0000","last_clean":"2026-02-22T20:52:20.159970+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:20.159970+0000","last_undegraded":"2026-02-22T20:52:20.159970+0000","last_fullsized":"2026-02-22T20:52:20.159970+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":14079,"reported_epoch":37,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:32.286781+0000","last_change":"2026-02-22T20:52:31.181416+0000","last_active":"2026-02-22T20:52:32.286781+0000","last_peered":"2026-02-22T20:52:32.286781+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:32.286781+0000","last_undegraded":"2026-02-22T20:52:32.286781+0000","last_fullsized":"2026-02-22T20:52:32.286781+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":18486,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1190,"num_bytes_recovered":1190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10110,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.154759+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:20.154759+0000","last_peered":"2026-02-22T20:52:20.154759+0000","last_clean":"2026-02-22T20:52:20.154759+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:20.154759+0000","last_undegraded":"2026-02-22T20:52:20.154759+0000","last_fullsized":"2026-02-22T20:52:20.154759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":33704,"reported_epoch":35,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:27.286558+0000","last_change":"2026-02-22T20:52:20.160858+0000","last_active":"2026-02-22T20:52:27.286558+0000","last_peered":"2026-02-22T20:52:27.286558+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:27.286558+0000","last_undegraded":"2026-02-22T20:52:27.286558+0000","last_fullsized":"2026-02-22T20:52:27.286558+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4694,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7823,"num_bytes_recovered":23124,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15966,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175795+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:31.175795+0000","last_peered":"2026-02-22T20:52:31.175795+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:31.175795+0000","last_undegraded":"2026-02-22T20:52:31.175795+0000","last_fullsized":"2026-02-22T20:52:31.175795+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10303,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175801+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:31.175801+0000","last_peered":"2026-02-22T20:52:31.175801+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:31.175801+0000","last_undegraded":"2026-02-22T20:52:31.175801+0000","last_fullsized":"2026-02-22T20:52:31.175801+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":94476,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32830,"num_bytes_recovered":124568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":865542144,"data_stored":944895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051598,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182417,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323736,"kb_used_data":134344,"kb_used_omap":20,"kb_used_meta":189355,"kb_avail":94048104,"statfs":{"total":96636764160,"available":96305258496,"internally_reserved":0,"allocated":137568256,"data_stored":248705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":193899792},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327440,"kb_used_data":138048,"kb_used_omap":20,"kb_used_meta":189355,"kb_avail":94044400,"statfs":{"total":96636764160,"available":96301465600,"internally_reserved":0,"allocated":141361152,"data_stored":249631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":193899792},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":141369344,"data_stored":141515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":141369344,"data_stored":141515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330313728,"data_stored":371004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":252489728,"data_stored":290861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:33.425 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:33.613 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:33.614 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:33.628 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":618,"stamp":"2026-02-22T20:52:33.120112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":94476,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32830,"num_bytes_recovered":124568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":651176,"kb_used_data":272392,"kb_used_omap":41,"kb_used_meta":378710,"kb_avail":188092504,"statfs":{"total":193273528320,"available":192606724096,"internally_reserved":0,"allocated":278929408,"data_stored":498336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42464,"internal_metadata":387799584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-16966,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8485,"num_bytes_recovered":8485,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000902"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15698,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.181207+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:31.181207+0000","last_peered":"2026-02-22T20:52:31.181207+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:31.181207+0000","last_undegraded":"2026-02-22T20:52:31.181207+0000","last_fullsized":"2026-02-22T20:52:31.181207+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15272,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175902+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:31.175902+0000","last_peered":"2026-02-22T20:52:31.175902+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:31.175902+0000","last_undegraded":"2026-02-22T20:52:31.175902+0000","last_fullsized":"2026-02-22T20:52:31.175902+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40489,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.159970+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:20.159970+0000","last_peered":"2026-02-22T20:52:20.159970+0000","last_clean":"2026-02-22T20:52:20.159970+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:20.159970+0000","last_undegraded":"2026-02-22T20:52:20.159970+0000","last_fullsized":"2026-02-22T20:52:20.159970+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":14079,"reported_epoch":37,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:32.286781+0000","last_change":"2026-02-22T20:52:31.181416+0000","last_active":"2026-02-22T20:52:32.286781+0000","last_peered":"2026-02-22T20:52:32.286781+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:32.286781+0000","last_undegraded":"2026-02-22T20:52:32.286781+0000","last_fullsized":"2026-02-22T20:52:32.286781+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":18486,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1190,"num_bytes_recovered":1190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10110,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.154759+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:20.154759+0000","last_peered":"2026-02-22T20:52:20.154759+0000","last_clean":"2026-02-22T20:52:20.154759+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:20.154759+0000","last_undegraded":"2026-02-22T20:52:20.154759+0000","last_fullsized":"2026-02-22T20:52:20.154759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":33704,"reported_epoch":35,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:27.286558+0000","last_change":"2026-02-22T20:52:20.160858+0000","last_active":"2026-02-22T20:52:27.286558+0000","last_peered":"2026-02-22T20:52:27.286558+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:27.286558+0000","last_undegraded":"2026-02-22T20:52:27.286558+0000","last_fullsized":"2026-02-22T20:52:27.286558+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4694,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7823,"num_bytes_recovered":23124,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15966,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175795+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:31.175795+0000","last_peered":"2026-02-22T20:52:31.175795+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:31.175795+0000","last_undegraded":"2026-02-22T20:52:31.175795+0000","last_fullsized":"2026-02-22T20:52:31.175795+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10303,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175801+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:31.175801+0000","last_peered":"2026-02-22T20:52:31.175801+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:31.175801+0000","last_undegraded":"2026-02-22T20:52:31.175801+0000","last_fullsized":"2026-02-22T20:52:31.175801+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":94476,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32830,"num_bytes_recovered":124568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":865542144,"data_stored":944895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051598,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182417,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323736,"kb_used_data":134344,"kb_used_omap":20,"kb_used_meta":189355,"kb_avail":94048104,"statfs":{"total":96636764160,"available":96305258496,"internally_reserved":0,"allocated":137568256,"data_stored":248705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":193899792},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327440,"kb_used_data":138048,"kb_used_omap":20,"kb_used_meta":189355,"kb_avail":94044400,"statfs":{"total":96636764160,"available":96301465600,"internally_reserved":0,"allocated":141361152,"data_stored":249631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":193899792},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":141369344,"data_stored":141515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":141369344,"data_stored":141515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330313728,"data_stored":371004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":252489728,"data_stored":290861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:33.629 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:33.629 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 15698, 'reported_epoch': 37, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:31.181207+0000', 'last_change': '2026-02-22T20:52:31.180859+0000', 'last_active': '2026-02-22T20:52:31.181207+0000', 'last_peered': '2026-02-22T20:52:31.181207+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:31.181207+0000', 'last_undegraded': '2026-02-22T20:52:31.181207+0000', 'last_fullsized': '2026-02-22T20:52:31.181207+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16792, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1810, 'num_bytes_recovered': 1810, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:33.629 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:33.629 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 15272, 'reported_epoch': 37, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:31.175902+0000', 'last_change': '2026-02-22T20:52:09.142836+0000', 'last_active': '2026-02-22T20:52:31.175902+0000', 'last_peered': '2026-02-22T20:52:31.175902+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:52:31.175902+0000', 'last_undegraded': '2026-02-22T20:52:31.175902+0000', 'last_fullsized': '2026-02-22T20:52:31.175902+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17314, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1626, 'num_bytes_recovered': 16909, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:33.629 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:52:33.629 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 14079, 'reported_epoch': 37, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:52:32.286781+0000', 'last_change': '2026-02-22T20:52:31.181416+0000', 'last_active': '2026-02-22T20:52:32.286781+0000', 'last_peered': '2026-02-22T20:52:32.286781+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:52:32.286781+0000', 'last_undegraded': '2026-02-22T20:52:32.286781+0000', 'last_fullsized': '2026-02-22T20:52:32.286781+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 18486, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1190, 'num_bytes_recovered': 1190, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:33.629 INFO:tasks.osd_recovery.ceph_manager:PG 1.2 is not active+clean 2026-02-22T20:52:33.629 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.2', 'version': "17'10169", 'reported_seq': 33704, 'reported_epoch': 35, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:52:27.286558+0000', 'last_change': '2026-02-22T20:52:20.160858+0000', 'last_active': '2026-02-22T20:52:27.286558+0000', 'last_peered': '2026-02-22T20:52:27.286558+0000', 'last_clean': '2026-02-22T20:50:31.136895+0000', 'last_became_active': '2026-02-22T20:51:26.182379+0000', 'last_became_peered': '2026-02-22T20:51:26.182379+0000', 'last_unstale': '2026-02-22T20:52:27.286558+0000', 'last_undegraded': '2026-02-22T20:52:27.286558+0000', 'last_fullsized': '2026-02-22T20:52:27.286558+0000', 'mapping_epoch': 30, 'log_start': "17'100", 'ondisk_log_start': "17'100", 'created': 10, 'last_epoch_clean': 17, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000577451, 'stat_sum': {'num_bytes': 40752, 'num_objects': 10168, 'num_object_clones': 0, 'num_object_copies': 20336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4694, 'num_objects_unfound': 0, 'num_objects_dirty': 10168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10169, 'num_write_kb': 10197, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7823, 'num_bytes_recovered': 23124, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10168}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:33.630 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:33.630 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 15966, 'reported_epoch': 37, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:31.175795+0000', 'last_change': '2026-02-22T20:52:20.160334+0000', 'last_active': '2026-02-22T20:52:31.175795+0000', 'last_peered': '2026-02-22T20:52:31.175795+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:31.175795+0000', 'last_undegraded': '2026-02-22T20:52:31.175795+0000', 'last_fullsized': '2026-02-22T20:52:31.175795+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16712, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1883, 'num_bytes_recovered': 1883, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:33.630 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:33.630 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10303, 'reported_epoch': 37, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:31.175801+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:52:31.175801+0000', 'last_peered': '2026-02-22T20:52:31.175801+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:31.175801+0000', 'last_undegraded': '2026-02-22T20:52:31.175801+0000', 'last_fullsized': '2026-02-22T20:52:31.175801+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:33.630 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:33.860 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:33.860 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1170,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":37,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":4},{"state_name":"active+clean","count":2},{"state_name":"active+remapped+backfilling","count":2}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":666804224,"bytes_avail":192606724096,"bytes_total":193273528320,"misplaced_objects":94476,"misplaced_total":163508,"misplaced_ratio":0.57780659050321692,"recovering_objects_per_sec":1060,"recovering_bytes_per_sec":1060,"recovering_keys_per_sec":0,"num_objects_recovered":8485,"num_bytes_recovered":8485,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:52:33.874 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:33.874 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:34.067 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:34.068 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:34.083 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":618,"stamp":"2026-02-22T20:52:33.120112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":94476,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32830,"num_bytes_recovered":124568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":651176,"kb_used_data":272392,"kb_used_omap":41,"kb_used_meta":378710,"kb_avail":188092504,"statfs":{"total":193273528320,"available":192606724096,"internally_reserved":0,"allocated":278929408,"data_stored":498336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42464,"internal_metadata":387799584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-16966,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8485,"num_bytes_recovered":8485,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000902"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15698,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.181207+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:31.181207+0000","last_peered":"2026-02-22T20:52:31.181207+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:31.181207+0000","last_undegraded":"2026-02-22T20:52:31.181207+0000","last_fullsized":"2026-02-22T20:52:31.181207+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15272,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175902+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:31.175902+0000","last_peered":"2026-02-22T20:52:31.175902+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:31.175902+0000","last_undegraded":"2026-02-22T20:52:31.175902+0000","last_fullsized":"2026-02-22T20:52:31.175902+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40489,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.159970+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:20.159970+0000","last_peered":"2026-02-22T20:52:20.159970+0000","last_clean":"2026-02-22T20:52:20.159970+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:20.159970+0000","last_undegraded":"2026-02-22T20:52:20.159970+0000","last_fullsized":"2026-02-22T20:52:20.159970+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":14079,"reported_epoch":37,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:32.286781+0000","last_change":"2026-02-22T20:52:31.181416+0000","last_active":"2026-02-22T20:52:32.286781+0000","last_peered":"2026-02-22T20:52:32.286781+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:32.286781+0000","last_undegraded":"2026-02-22T20:52:32.286781+0000","last_fullsized":"2026-02-22T20:52:32.286781+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":18486,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1190,"num_bytes_recovered":1190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10110,"reported_epoch":35,"state":"active+clean","last_fresh":"2026-02-22T20:52:20.154759+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:20.154759+0000","last_peered":"2026-02-22T20:52:20.154759+0000","last_clean":"2026-02-22T20:52:20.154759+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:20.154759+0000","last_undegraded":"2026-02-22T20:52:20.154759+0000","last_fullsized":"2026-02-22T20:52:20.154759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":33704,"reported_epoch":35,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:27.286558+0000","last_change":"2026-02-22T20:52:20.160858+0000","last_active":"2026-02-22T20:52:27.286558+0000","last_peered":"2026-02-22T20:52:27.286558+0000","last_clean":"2026-02-22T20:50:31.136895+0000","last_became_active":"2026-02-22T20:51:26.182379+0000","last_became_peered":"2026-02-22T20:51:26.182379+0000","last_unstale":"2026-02-22T20:52:27.286558+0000","last_undegraded":"2026-02-22T20:52:27.286558+0000","last_fullsized":"2026-02-22T20:52:27.286558+0000","mapping_epoch":30,"log_start":"17'100","ondisk_log_start":"17'100","created":10,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4694,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7823,"num_bytes_recovered":23124,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10168}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15966,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175795+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:31.175795+0000","last_peered":"2026-02-22T20:52:31.175795+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:31.175795+0000","last_undegraded":"2026-02-22T20:52:31.175795+0000","last_fullsized":"2026-02-22T20:52:31.175795+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10303,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175801+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:31.175801+0000","last_peered":"2026-02-22T20:52:31.175801+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:31.175801+0000","last_undegraded":"2026-02-22T20:52:31.175801+0000","last_fullsized":"2026-02-22T20:52:31.175801+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":94476,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32830,"num_bytes_recovered":124568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":865542144,"data_stored":944895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80254,"ondisk_log_size":80254,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051598,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182417,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705902,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323736,"kb_used_data":134344,"kb_used_omap":20,"kb_used_meta":189355,"kb_avail":94048104,"statfs":{"total":96636764160,"available":96305258496,"internally_reserved":0,"allocated":137568256,"data_stored":248705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":193899792},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327440,"kb_used_data":138048,"kb_used_omap":20,"kb_used_meta":189355,"kb_avail":94044400,"statfs":{"total":96636764160,"available":96301465600,"internally_reserved":0,"allocated":141361152,"data_stored":249631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21232,"internal_metadata":193899792},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":141369344,"data_stored":141515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":141369344,"data_stored":141515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330313728,"data_stored":371004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":252489728,"data_stored":290861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:37.085 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:37.277 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:37.278 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:37.291 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":620,"stamp":"2026-02-22T20:52:37.120570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":89782,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35176,"num_bytes_recovered":142215,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":766196,"kb_used_data":315220,"kb_used_omap":44,"kb_used_meta":450899,"kb_avail":187977484,"statfs":{"total":193273528320,"available":192488943616,"internally_reserved":0,"allocated":322785280,"data_stored":537697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":45068,"internal_metadata":461721588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10690,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5346,"num_bytes_recovered":20647,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000929"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15698,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.181207+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:31.181207+0000","last_peered":"2026-02-22T20:52:31.181207+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:31.181207+0000","last_undegraded":"2026-02-22T20:52:31.181207+0000","last_fullsized":"2026-02-22T20:52:31.181207+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15272,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175902+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:31.175902+0000","last_peered":"2026-02-22T20:52:31.175902+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:31.175902+0000","last_undegraded":"2026-02-22T20:52:31.175902+0000","last_fullsized":"2026-02-22T20:52:31.175902+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40493,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.174338+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:31.174338+0000","last_peered":"2026-02-22T20:52:31.174338+0000","last_clean":"2026-02-22T20:52:31.174338+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:31.174338+0000","last_undegraded":"2026-02-22T20:52:31.174338+0000","last_fullsized":"2026-02-22T20:52:31.174338+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":14079,"reported_epoch":37,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:32.286781+0000","last_change":"2026-02-22T20:52:31.181416+0000","last_active":"2026-02-22T20:52:32.286781+0000","last_peered":"2026-02-22T20:52:32.286781+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:32.286781+0000","last_undegraded":"2026-02-22T20:52:32.286781+0000","last_fullsized":"2026-02-22T20:52:32.286781+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":18486,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1190,"num_bytes_recovered":1190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10114,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.175297+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:31.175297+0000","last_peered":"2026-02-22T20:52:31.175297+0000","last_clean":"2026-02-22T20:52:31.175297+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:31.175297+0000","last_undegraded":"2026-02-22T20:52:31.175297+0000","last_fullsized":"2026-02-22T20:52:31.175297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40753,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.180417+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:31.180417+0000","last_peered":"2026-02-22T20:52:31.180417+0000","last_clean":"2026-02-22T20:52:31.180417+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:31.180417+0000","last_undegraded":"2026-02-22T20:52:31.180417+0000","last_fullsized":"2026-02-22T20:52:31.180417+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15966,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175795+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:31.175795+0000","last_peered":"2026-02-22T20:52:31.175795+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:31.175795+0000","last_undegraded":"2026-02-22T20:52:31.175795+0000","last_fullsized":"2026-02-22T20:52:31.175795+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10303,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175801+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:31.175801+0000","last_peered":"2026-02-22T20:52:31.175801+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:31.175801+0000","last_undegraded":"2026-02-22T20:52:31.175801+0000","last_fullsized":"2026-02-22T20:52:31.175801+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":89782,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35176,"num_bytes_recovered":142215,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":867721216,"data_stored":945409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051599,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182417,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381508,"kb_used_data":156020,"kb_used_omap":22,"kb_used_meta":225449,"kb_avail":93990332,"statfs":{"total":96636764160,"available":96246099968,"internally_reserved":0,"allocated":159764480,"data_stored":260811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":230860794},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384688,"kb_used_data":159200,"kb_used_omap":22,"kb_used_meta":225449,"kb_avail":93987152,"statfs":{"total":96636764160,"available":96242843648,"internally_reserved":0,"allocated":163020800,"data_stored":276886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":230860794},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163295232,"data_stored":162148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163295232,"data_stored":162148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330313728,"data_stored":371004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":210817024,"data_stored":250109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:37.293 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:37.484 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:37.485 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:37.499 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":620,"stamp":"2026-02-22T20:52:37.120570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":89782,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35176,"num_bytes_recovered":142215,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":766196,"kb_used_data":315220,"kb_used_omap":44,"kb_used_meta":450899,"kb_avail":187977484,"statfs":{"total":193273528320,"available":192488943616,"internally_reserved":0,"allocated":322785280,"data_stored":537697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":45068,"internal_metadata":461721588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10690,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5346,"num_bytes_recovered":20647,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000929"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15698,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.181207+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:31.181207+0000","last_peered":"2026-02-22T20:52:31.181207+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:31.181207+0000","last_undegraded":"2026-02-22T20:52:31.181207+0000","last_fullsized":"2026-02-22T20:52:31.181207+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15272,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175902+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:31.175902+0000","last_peered":"2026-02-22T20:52:31.175902+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:31.175902+0000","last_undegraded":"2026-02-22T20:52:31.175902+0000","last_fullsized":"2026-02-22T20:52:31.175902+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40493,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.174338+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:31.174338+0000","last_peered":"2026-02-22T20:52:31.174338+0000","last_clean":"2026-02-22T20:52:31.174338+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:31.174338+0000","last_undegraded":"2026-02-22T20:52:31.174338+0000","last_fullsized":"2026-02-22T20:52:31.174338+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":14079,"reported_epoch":37,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:32.286781+0000","last_change":"2026-02-22T20:52:31.181416+0000","last_active":"2026-02-22T20:52:32.286781+0000","last_peered":"2026-02-22T20:52:32.286781+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:32.286781+0000","last_undegraded":"2026-02-22T20:52:32.286781+0000","last_fullsized":"2026-02-22T20:52:32.286781+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":18486,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1190,"num_bytes_recovered":1190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10114,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.175297+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:31.175297+0000","last_peered":"2026-02-22T20:52:31.175297+0000","last_clean":"2026-02-22T20:52:31.175297+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:31.175297+0000","last_undegraded":"2026-02-22T20:52:31.175297+0000","last_fullsized":"2026-02-22T20:52:31.175297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40753,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.180417+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:31.180417+0000","last_peered":"2026-02-22T20:52:31.180417+0000","last_clean":"2026-02-22T20:52:31.180417+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:31.180417+0000","last_undegraded":"2026-02-22T20:52:31.180417+0000","last_fullsized":"2026-02-22T20:52:31.180417+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15966,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175795+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:31.175795+0000","last_peered":"2026-02-22T20:52:31.175795+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:31.175795+0000","last_undegraded":"2026-02-22T20:52:31.175795+0000","last_fullsized":"2026-02-22T20:52:31.175795+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10303,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175801+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:31.175801+0000","last_peered":"2026-02-22T20:52:31.175801+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:31.175801+0000","last_undegraded":"2026-02-22T20:52:31.175801+0000","last_fullsized":"2026-02-22T20:52:31.175801+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":89782,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35176,"num_bytes_recovered":142215,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":867721216,"data_stored":945409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051599,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182417,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381508,"kb_used_data":156020,"kb_used_omap":22,"kb_used_meta":225449,"kb_avail":93990332,"statfs":{"total":96636764160,"available":96246099968,"internally_reserved":0,"allocated":159764480,"data_stored":260811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":230860794},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384688,"kb_used_data":159200,"kb_used_omap":22,"kb_used_meta":225449,"kb_avail":93987152,"statfs":{"total":96636764160,"available":96242843648,"internally_reserved":0,"allocated":163020800,"data_stored":276886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":230860794},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163295232,"data_stored":162148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163295232,"data_stored":162148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330313728,"data_stored":371004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":210817024,"data_stored":250109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:37.500 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:37.501 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 15698, 'reported_epoch': 37, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:31.181207+0000', 'last_change': '2026-02-22T20:52:31.180859+0000', 'last_active': '2026-02-22T20:52:31.181207+0000', 'last_peered': '2026-02-22T20:52:31.181207+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:31.181207+0000', 'last_undegraded': '2026-02-22T20:52:31.181207+0000', 'last_fullsized': '2026-02-22T20:52:31.181207+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16792, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1810, 'num_bytes_recovered': 1810, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:37.501 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:37.501 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 15272, 'reported_epoch': 37, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:31.175902+0000', 'last_change': '2026-02-22T20:52:09.142836+0000', 'last_active': '2026-02-22T20:52:31.175902+0000', 'last_peered': '2026-02-22T20:52:31.175902+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:52:31.175902+0000', 'last_undegraded': '2026-02-22T20:52:31.175902+0000', 'last_fullsized': '2026-02-22T20:52:31.175902+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17314, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1626, 'num_bytes_recovered': 16909, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:37.501 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:52:37.501 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 14079, 'reported_epoch': 37, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:52:32.286781+0000', 'last_change': '2026-02-22T20:52:31.181416+0000', 'last_active': '2026-02-22T20:52:32.286781+0000', 'last_peered': '2026-02-22T20:52:32.286781+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:52:32.286781+0000', 'last_undegraded': '2026-02-22T20:52:32.286781+0000', 'last_fullsized': '2026-02-22T20:52:32.286781+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 18486, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1190, 'num_bytes_recovered': 1190, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:37.502 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:37.502 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 15966, 'reported_epoch': 37, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:31.175795+0000', 'last_change': '2026-02-22T20:52:20.160334+0000', 'last_active': '2026-02-22T20:52:31.175795+0000', 'last_peered': '2026-02-22T20:52:31.175795+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:31.175795+0000', 'last_undegraded': '2026-02-22T20:52:31.175795+0000', 'last_fullsized': '2026-02-22T20:52:31.175795+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16712, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1883, 'num_bytes_recovered': 1883, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:37.502 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:37.502 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10303, 'reported_epoch': 37, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:31.175801+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:52:31.175801+0000', 'last_peered': '2026-02-22T20:52:31.175801+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:31.175801+0000', 'last_undegraded': '2026-02-22T20:52:31.175801+0000', 'last_fullsized': '2026-02-22T20:52:31.175801+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:37.502 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:37.733 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:37.733 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1174,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":37,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":4},{"state_name":"active+clean","count":3},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":784584704,"bytes_avail":192488943616,"bytes_total":193273528320,"misplaced_objects":89782,"misplaced_total":163508,"misplaced_ratio":0.54909851505736718,"recovering_objects_per_sec":668,"recovering_bytes_per_sec":2580,"recovering_keys_per_sec":0,"num_objects_recovered":5346,"num_bytes_recovered":20647,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:52:37.749 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:37.749 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:37.935 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:37.936 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:37.949 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":620,"stamp":"2026-02-22T20:52:37.120570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":89782,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35176,"num_bytes_recovered":142215,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":766196,"kb_used_data":315220,"kb_used_omap":44,"kb_used_meta":450899,"kb_avail":187977484,"statfs":{"total":193273528320,"available":192488943616,"internally_reserved":0,"allocated":322785280,"data_stored":537697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":45068,"internal_metadata":461721588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10690,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5346,"num_bytes_recovered":20647,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000929"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15698,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.181207+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:31.181207+0000","last_peered":"2026-02-22T20:52:31.181207+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:31.181207+0000","last_undegraded":"2026-02-22T20:52:31.181207+0000","last_fullsized":"2026-02-22T20:52:31.181207+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15272,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175902+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:31.175902+0000","last_peered":"2026-02-22T20:52:31.175902+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:31.175902+0000","last_undegraded":"2026-02-22T20:52:31.175902+0000","last_fullsized":"2026-02-22T20:52:31.175902+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40493,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.174338+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:31.174338+0000","last_peered":"2026-02-22T20:52:31.174338+0000","last_clean":"2026-02-22T20:52:31.174338+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:31.174338+0000","last_undegraded":"2026-02-22T20:52:31.174338+0000","last_fullsized":"2026-02-22T20:52:31.174338+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":14079,"reported_epoch":37,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:32.286781+0000","last_change":"2026-02-22T20:52:31.181416+0000","last_active":"2026-02-22T20:52:32.286781+0000","last_peered":"2026-02-22T20:52:32.286781+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:32.286781+0000","last_undegraded":"2026-02-22T20:52:32.286781+0000","last_fullsized":"2026-02-22T20:52:32.286781+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":18486,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1190,"num_bytes_recovered":1190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10114,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.175297+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:31.175297+0000","last_peered":"2026-02-22T20:52:31.175297+0000","last_clean":"2026-02-22T20:52:31.175297+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:31.175297+0000","last_undegraded":"2026-02-22T20:52:31.175297+0000","last_fullsized":"2026-02-22T20:52:31.175297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40753,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.180417+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:31.180417+0000","last_peered":"2026-02-22T20:52:31.180417+0000","last_clean":"2026-02-22T20:52:31.180417+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:31.180417+0000","last_undegraded":"2026-02-22T20:52:31.180417+0000","last_fullsized":"2026-02-22T20:52:31.180417+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15966,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175795+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:31.175795+0000","last_peered":"2026-02-22T20:52:31.175795+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:31.175795+0000","last_undegraded":"2026-02-22T20:52:31.175795+0000","last_fullsized":"2026-02-22T20:52:31.175795+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10303,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175801+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:31.175801+0000","last_peered":"2026-02-22T20:52:31.175801+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:31.175801+0000","last_undegraded":"2026-02-22T20:52:31.175801+0000","last_fullsized":"2026-02-22T20:52:31.175801+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":89782,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35176,"num_bytes_recovered":142215,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":867721216,"data_stored":945409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051599,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182417,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381508,"kb_used_data":156020,"kb_used_omap":22,"kb_used_meta":225449,"kb_avail":93990332,"statfs":{"total":96636764160,"available":96246099968,"internally_reserved":0,"allocated":159764480,"data_stored":260811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":230860794},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384688,"kb_used_data":159200,"kb_used_omap":22,"kb_used_meta":225449,"kb_avail":93987152,"statfs":{"total":96636764160,"available":96242843648,"internally_reserved":0,"allocated":163020800,"data_stored":276886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":230860794},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163295232,"data_stored":162148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163295232,"data_stored":162148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330313728,"data_stored":371004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":210817024,"data_stored":250109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:40.950 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:41.142 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:41.142 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:41.155 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":622,"stamp":"2026-02-22T20:52:41.120999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":78848,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40643,"num_bytes_recovered":162965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":881968,"kb_used_data":358800,"kb_used_omap":44,"kb_used_meta":523091,"kb_avail":187861712,"statfs":{"total":193273528320,"available":192370393088,"internally_reserved":0,"allocated":367411200,"data_stored":624992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":45068,"internal_metadata":535646196},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21624,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10813,"num_bytes_recovered":41397,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001117"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15698,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.181207+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:31.181207+0000","last_peered":"2026-02-22T20:52:31.181207+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:31.181207+0000","last_undegraded":"2026-02-22T20:52:31.181207+0000","last_fullsized":"2026-02-22T20:52:31.181207+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15272,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175902+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:31.175902+0000","last_peered":"2026-02-22T20:52:31.175902+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:31.175902+0000","last_undegraded":"2026-02-22T20:52:31.175902+0000","last_fullsized":"2026-02-22T20:52:31.175902+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40493,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.174338+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:31.174338+0000","last_peered":"2026-02-22T20:52:31.174338+0000","last_clean":"2026-02-22T20:52:31.174338+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:31.174338+0000","last_undegraded":"2026-02-22T20:52:31.174338+0000","last_fullsized":"2026-02-22T20:52:31.174338+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":30480,"reported_epoch":37,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:37.287387+0000","last_change":"2026-02-22T20:52:31.181416+0000","last_active":"2026-02-22T20:52:37.287387+0000","last_peered":"2026-02-22T20:52:37.287387+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:37.287387+0000","last_undegraded":"2026-02-22T20:52:37.287387+0000","last_fullsized":"2026-02-22T20:52:37.287387+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7552,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6657,"num_bytes_recovered":21940,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10114,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.175297+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:31.175297+0000","last_peered":"2026-02-22T20:52:31.175297+0000","last_clean":"2026-02-22T20:52:31.175297+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:31.175297+0000","last_undegraded":"2026-02-22T20:52:31.175297+0000","last_fullsized":"2026-02-22T20:52:31.175297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40753,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.180417+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:31.180417+0000","last_peered":"2026-02-22T20:52:31.180417+0000","last_clean":"2026-02-22T20:52:31.180417+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:31.180417+0000","last_undegraded":"2026-02-22T20:52:31.180417+0000","last_fullsized":"2026-02-22T20:52:31.180417+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15966,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175795+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:31.175795+0000","last_peered":"2026-02-22T20:52:31.175795+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:31.175795+0000","last_undegraded":"2026-02-22T20:52:31.175795+0000","last_fullsized":"2026-02-22T20:52:31.175795+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10303,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175801+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:31.175801+0000","last_peered":"2026-02-22T20:52:31.175801+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:31.175801+0000","last_undegraded":"2026-02-22T20:52:31.175801+0000","last_fullsized":"2026-02-22T20:52:31.175801+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":78848,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40643,"num_bytes_recovered":162965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":910614528,"data_stored":1047561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051600,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182418,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439164,"kb_used_data":177580,"kb_used_omap":22,"kb_used_meta":261545,"kb_avail":93932676,"statfs":{"total":96636764160,"available":96187060224,"internally_reserved":0,"allocated":181841920,"data_stored":296761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":267823098},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442804,"kb_used_data":181220,"kb_used_omap":22,"kb_used_meta":261545,"kb_avail":93929036,"statfs":{"total":96636764160,"available":96183332864,"internally_reserved":0,"allocated":185569280,"data_stored":328231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":267823098},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185663488,"data_stored":213449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185663488,"data_stored":213449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":328470528,"data_stored":370554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":210817024,"data_stored":250109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:41.156 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:41.348 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:41.348 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:41.362 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":623,"stamp":"2026-02-22T20:52:41.186088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":78848,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40643,"num_bytes_recovered":162965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":881968,"kb_used_data":358800,"kb_used_omap":44,"kb_used_meta":523091,"kb_avail":187861712,"statfs":{"total":193273528320,"available":192370393088,"internally_reserved":0,"allocated":367411200,"data_stored":624992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":45068,"internal_metadata":535646196},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21624,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10813,"num_bytes_recovered":41397,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.013774"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15698,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.181207+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:31.181207+0000","last_peered":"2026-02-22T20:52:31.181207+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:31.181207+0000","last_undegraded":"2026-02-22T20:52:31.181207+0000","last_fullsized":"2026-02-22T20:52:31.181207+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15272,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175902+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:31.175902+0000","last_peered":"2026-02-22T20:52:31.175902+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:31.175902+0000","last_undegraded":"2026-02-22T20:52:31.175902+0000","last_fullsized":"2026-02-22T20:52:31.175902+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40493,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.174338+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:31.174338+0000","last_peered":"2026-02-22T20:52:31.174338+0000","last_clean":"2026-02-22T20:52:31.174338+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:31.174338+0000","last_undegraded":"2026-02-22T20:52:31.174338+0000","last_fullsized":"2026-02-22T20:52:31.174338+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":30480,"reported_epoch":37,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:37.287387+0000","last_change":"2026-02-22T20:52:31.181416+0000","last_active":"2026-02-22T20:52:37.287387+0000","last_peered":"2026-02-22T20:52:37.287387+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:37.287387+0000","last_undegraded":"2026-02-22T20:52:37.287387+0000","last_fullsized":"2026-02-22T20:52:37.287387+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7552,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6657,"num_bytes_recovered":21940,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10114,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.175297+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:31.175297+0000","last_peered":"2026-02-22T20:52:31.175297+0000","last_clean":"2026-02-22T20:52:31.175297+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:31.175297+0000","last_undegraded":"2026-02-22T20:52:31.175297+0000","last_fullsized":"2026-02-22T20:52:31.175297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40753,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.180417+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:31.180417+0000","last_peered":"2026-02-22T20:52:31.180417+0000","last_clean":"2026-02-22T20:52:31.180417+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:31.180417+0000","last_undegraded":"2026-02-22T20:52:31.180417+0000","last_fullsized":"2026-02-22T20:52:31.180417+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15966,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175795+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:31.175795+0000","last_peered":"2026-02-22T20:52:31.175795+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:31.175795+0000","last_undegraded":"2026-02-22T20:52:31.175795+0000","last_fullsized":"2026-02-22T20:52:31.175795+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10303,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175801+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:31.175801+0000","last_peered":"2026-02-22T20:52:31.175801+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:31.175801+0000","last_undegraded":"2026-02-22T20:52:31.175801+0000","last_fullsized":"2026-02-22T20:52:31.175801+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":78848,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40643,"num_bytes_recovered":162965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":910614528,"data_stored":1047561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051600,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182418,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439164,"kb_used_data":177580,"kb_used_omap":22,"kb_used_meta":261545,"kb_avail":93932676,"statfs":{"total":96636764160,"available":96187060224,"internally_reserved":0,"allocated":181841920,"data_stored":296761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":267823098},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442804,"kb_used_data":181220,"kb_used_omap":22,"kb_used_meta":261545,"kb_avail":93929036,"statfs":{"total":96636764160,"available":96183332864,"internally_reserved":0,"allocated":185569280,"data_stored":328231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":267823098},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185663488,"data_stored":213449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185663488,"data_stored":213449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":328470528,"data_stored":370554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":210817024,"data_stored":250109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:41.363 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:41.363 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 15698, 'reported_epoch': 37, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:31.181207+0000', 'last_change': '2026-02-22T20:52:31.180859+0000', 'last_active': '2026-02-22T20:52:31.181207+0000', 'last_peered': '2026-02-22T20:52:31.181207+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:31.181207+0000', 'last_undegraded': '2026-02-22T20:52:31.181207+0000', 'last_fullsized': '2026-02-22T20:52:31.181207+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16792, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1810, 'num_bytes_recovered': 1810, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:41.363 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:41.364 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 15272, 'reported_epoch': 37, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:31.175902+0000', 'last_change': '2026-02-22T20:52:09.142836+0000', 'last_active': '2026-02-22T20:52:31.175902+0000', 'last_peered': '2026-02-22T20:52:31.175902+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:52:31.175902+0000', 'last_undegraded': '2026-02-22T20:52:31.175902+0000', 'last_fullsized': '2026-02-22T20:52:31.175902+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17314, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1626, 'num_bytes_recovered': 16909, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:41.364 INFO:tasks.osd_recovery.ceph_manager:PG 1.4 is not active+clean 2026-02-22T20:52:41.364 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.4', 'version': "17'10432", 'reported_seq': 30480, 'reported_epoch': 37, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:52:37.287387+0000', 'last_change': '2026-02-22T20:52:31.181416+0000', 'last_active': '2026-02-22T20:52:37.287387+0000', 'last_peered': '2026-02-22T20:52:37.287387+0000', 'last_clean': '2026-02-22T20:50:31.054798+0000', 'last_became_active': '2026-02-22T20:51:26.182450+0000', 'last_became_peered': '2026-02-22T20:51:26.182450+0000', 'last_unstale': '2026-02-22T20:52:37.287387+0000', 'last_undegraded': '2026-02-22T20:52:37.287387+0000', 'last_fullsized': '2026-02-22T20:52:37.287387+0000', 'mapping_epoch': 30, 'log_start': "17'400", 'ondisk_log_start': "17'400", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000415801, 'stat_sum': {'num_bytes': 71564, 'num_objects': 10432, 'num_object_clones': 0, 'num_object_copies': 20864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 7552, 'num_objects_unfound': 0, 'num_objects_dirty': 10432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10432, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6657, 'num_bytes_recovered': 21940, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:41.364 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:41.364 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 15966, 'reported_epoch': 37, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:31.175795+0000', 'last_change': '2026-02-22T20:52:20.160334+0000', 'last_active': '2026-02-22T20:52:31.175795+0000', 'last_peered': '2026-02-22T20:52:31.175795+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:31.175795+0000', 'last_undegraded': '2026-02-22T20:52:31.175795+0000', 'last_fullsized': '2026-02-22T20:52:31.175795+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16712, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1883, 'num_bytes_recovered': 1883, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:41.364 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:41.364 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 10303, 'reported_epoch': 37, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:31.175801+0000', 'last_change': '2026-02-22T20:51:26.190780+0000', 'last_active': '2026-02-22T20:52:31.175801+0000', 'last_peered': '2026-02-22T20:52:31.175801+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:31.175801+0000', 'last_undegraded': '2026-02-22T20:52:31.175801+0000', 'last_fullsized': '2026-02-22T20:52:31.175801+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 20478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:41.365 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:41.599 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:41.599 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1178,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":38,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+remapped+backfill_wait","count":4},{"state_name":"active+clean","count":3},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":903135232,"bytes_avail":192370393088,"bytes_total":193273528320,"misplaced_objects":78848,"misplaced_total":163508,"misplaced_ratio":0.48222716931281651,"recovering_objects_per_sec":1081,"recovering_bytes_per_sec":4139,"recovering_keys_per_sec":0,"num_objects_recovered":10813,"num_bytes_recovered":41397,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:52:41.613 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:41.613 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:41.831 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:41.831 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:41.845 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":623,"stamp":"2026-02-22T20:52:41.186088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":78848,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40643,"num_bytes_recovered":162965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":881968,"kb_used_data":358800,"kb_used_omap":44,"kb_used_meta":523091,"kb_avail":187861712,"statfs":{"total":193273528320,"available":192370393088,"internally_reserved":0,"allocated":367411200,"data_stored":624992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":45068,"internal_metadata":535646196},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21624,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10813,"num_bytes_recovered":41397,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.013774"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15698,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.181207+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:31.181207+0000","last_peered":"2026-02-22T20:52:31.181207+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:31.181207+0000","last_undegraded":"2026-02-22T20:52:31.181207+0000","last_fullsized":"2026-02-22T20:52:31.181207+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15272,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175902+0000","last_change":"2026-02-22T20:52:09.142836+0000","last_active":"2026-02-22T20:52:31.175902+0000","last_peered":"2026-02-22T20:52:31.175902+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:31.175902+0000","last_undegraded":"2026-02-22T20:52:31.175902+0000","last_fullsized":"2026-02-22T20:52:31.175902+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17314,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1626,"num_bytes_recovered":16909,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40493,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.174338+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:31.174338+0000","last_peered":"2026-02-22T20:52:31.174338+0000","last_clean":"2026-02-22T20:52:31.174338+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:31.174338+0000","last_undegraded":"2026-02-22T20:52:31.174338+0000","last_fullsized":"2026-02-22T20:52:31.174338+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":30480,"reported_epoch":37,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:37.287387+0000","last_change":"2026-02-22T20:52:31.181416+0000","last_active":"2026-02-22T20:52:37.287387+0000","last_peered":"2026-02-22T20:52:37.287387+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:37.287387+0000","last_undegraded":"2026-02-22T20:52:37.287387+0000","last_fullsized":"2026-02-22T20:52:37.287387+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7552,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6657,"num_bytes_recovered":21940,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10114,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.175297+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:31.175297+0000","last_peered":"2026-02-22T20:52:31.175297+0000","last_clean":"2026-02-22T20:52:31.175297+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:31.175297+0000","last_undegraded":"2026-02-22T20:52:31.175297+0000","last_fullsized":"2026-02-22T20:52:31.175297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40753,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.180417+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:31.180417+0000","last_peered":"2026-02-22T20:52:31.180417+0000","last_clean":"2026-02-22T20:52:31.180417+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:31.180417+0000","last_undegraded":"2026-02-22T20:52:31.180417+0000","last_fullsized":"2026-02-22T20:52:31.180417+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15966,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175795+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:31.175795+0000","last_peered":"2026-02-22T20:52:31.175795+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:31.175795+0000","last_undegraded":"2026-02-22T20:52:31.175795+0000","last_fullsized":"2026-02-22T20:52:31.175795+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":10303,"reported_epoch":37,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:31.175801+0000","last_change":"2026-02-22T20:51:26.190780+0000","last_active":"2026-02-22T20:52:31.175801+0000","last_peered":"2026-02-22T20:52:31.175801+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:31.175801+0000","last_undegraded":"2026-02-22T20:52:31.175801+0000","last_fullsized":"2026-02-22T20:52:31.175801+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":78848,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40643,"num_bytes_recovered":162965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":910614528,"data_stored":1047561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051600,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182418,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439164,"kb_used_data":177580,"kb_used_omap":22,"kb_used_meta":261545,"kb_avail":93932676,"statfs":{"total":96636764160,"available":96187060224,"internally_reserved":0,"allocated":181841920,"data_stored":296761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":267823098},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442804,"kb_used_data":181220,"kb_used_omap":22,"kb_used_meta":261545,"kb_avail":93929036,"statfs":{"total":96636764160,"available":96183332864,"internally_reserved":0,"allocated":185569280,"data_stored":328231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":267823098},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185663488,"data_stored":213449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185663488,"data_stored":213449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":328470528,"data_stored":370554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":210817024,"data_stored":250109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:44.846 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:45.035 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:45.035 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:45.049 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":625,"stamp":"2026-02-22T20:52:43.121216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":75660,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42238,"num_bytes_recovered":164560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":881968,"kb_used_data":358800,"kb_used_omap":44,"kb_used_meta":523091,"kb_avail":187861712,"statfs":{"total":193273528320,"available":192370393088,"internally_reserved":0,"allocated":367411200,"data_stored":624992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":45068,"internal_metadata":535646196},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-14122,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7062,"num_bytes_recovered":22345,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000873"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15702,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189892+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:42.189892+0000","last_peered":"2026-02-22T20:52:42.189892+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:42.189892+0000","last_undegraded":"2026-02-22T20:52:42.189892+0000","last_fullsized":"2026-02-22T20:52:42.189892+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15560,"reported_epoch":39,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:42.287698+0000","last_change":"2026-02-22T20:52:42.197046+0000","last_active":"2026-02-22T20:52:42.287698+0000","last_peered":"2026-02-22T20:52:42.287698+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:42.287698+0000","last_undegraded":"2026-02-22T20:52:42.287698+0000","last_fullsized":"2026-02-22T20:52:42.287698+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17126,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1720,"num_bytes_recovered":17003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40493,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.174338+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:31.174338+0000","last_peered":"2026-02-22T20:52:31.174338+0000","last_clean":"2026-02-22T20:52:31.174338+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:31.174338+0000","last_undegraded":"2026-02-22T20:52:31.174338+0000","last_fullsized":"2026-02-22T20:52:31.174338+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":30480,"reported_epoch":37,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:37.287387+0000","last_change":"2026-02-22T20:52:31.181416+0000","last_active":"2026-02-22T20:52:37.287387+0000","last_peered":"2026-02-22T20:52:37.287387+0000","last_clean":"2026-02-22T20:50:31.054798+0000","last_became_active":"2026-02-22T20:51:26.182450+0000","last_became_peered":"2026-02-22T20:51:26.182450+0000","last_unstale":"2026-02-22T20:52:37.287387+0000","last_undegraded":"2026-02-22T20:52:37.287387+0000","last_fullsized":"2026-02-22T20:52:37.287387+0000","mapping_epoch":30,"log_start":"17'400","ondisk_log_start":"17'400","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7552,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6657,"num_bytes_recovered":21940,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10432}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10114,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.175297+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:31.175297+0000","last_peered":"2026-02-22T20:52:31.175297+0000","last_clean":"2026-02-22T20:52:31.175297+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:31.175297+0000","last_undegraded":"2026-02-22T20:52:31.175297+0000","last_fullsized":"2026-02-22T20:52:31.175297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40753,"reported_epoch":37,"state":"active+clean","last_fresh":"2026-02-22T20:52:31.180417+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:31.180417+0000","last_peered":"2026-02-22T20:52:31.180417+0000","last_clean":"2026-02-22T20:52:31.180417+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:31.180417+0000","last_undegraded":"2026-02-22T20:52:31.180417+0000","last_fullsized":"2026-02-22T20:52:31.180417+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15970,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189827+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:42.189827+0000","last_peered":"2026-02-22T20:52:42.189827+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:42.189827+0000","last_undegraded":"2026-02-22T20:52:42.189827+0000","last_fullsized":"2026-02-22T20:52:42.189827+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14814,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.197002+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:42.197002+0000","last_peered":"2026-02-22T20:52:42.197002+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:42.197002+0000","last_undegraded":"2026-02-22T20:52:42.197002+0000","last_fullsized":"2026-02-22T20:52:42.197002+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":75660,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42238,"num_bytes_recovered":164560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":908640256,"data_stored":1031799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051600,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182419,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705904,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439164,"kb_used_data":177580,"kb_used_omap":22,"kb_used_meta":261545,"kb_avail":93932676,"statfs":{"total":96636764160,"available":96187060224,"internally_reserved":0,"allocated":181841920,"data_stored":296761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":267823098},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442804,"kb_used_data":181220,"kb_used_omap":22,"kb_used_meta":261545,"kb_avail":93929036,"statfs":{"total":96636764160,"available":96183332864,"internally_reserved":0,"allocated":185569280,"data_stored":328231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22534,"internal_metadata":267823098},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185663488,"data_stored":213449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185663488,"data_stored":213449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326496256,"data_stored":354792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":210817024,"data_stored":250109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:45.050 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:45.239 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:45.239 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:45.251 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":626,"stamp":"2026-02-22T20:52:45.121445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":68108,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46013,"num_bytes_recovered":214184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":943088,"kb_used_data":401872,"kb_used_omap":46,"kb_used_meta":541137,"kb_avail":187800592,"statfs":{"total":193273528320,"available":192307806208,"internally_reserved":0,"allocated":411516928,"data_stored":740698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47672,"internal_metadata":554124744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21674,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10837,"num_bytes_recovered":71969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000875"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15702,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189892+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:42.189892+0000","last_peered":"2026-02-22T20:52:42.189892+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:42.189892+0000","last_undegraded":"2026-02-22T20:52:42.189892+0000","last_fullsized":"2026-02-22T20:52:42.189892+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15560,"reported_epoch":39,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:42.287698+0000","last_change":"2026-02-22T20:52:42.197046+0000","last_active":"2026-02-22T20:52:42.287698+0000","last_peered":"2026-02-22T20:52:42.287698+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:42.287698+0000","last_undegraded":"2026-02-22T20:52:42.287698+0000","last_fullsized":"2026-02-22T20:52:42.287698+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17126,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1720,"num_bytes_recovered":17003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40497,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190685+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:42.190685+0000","last_peered":"2026-02-22T20:52:42.190685+0000","last_clean":"2026-02-22T20:52:42.190685+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:42.190685+0000","last_undegraded":"2026-02-22T20:52:42.190685+0000","last_fullsized":"2026-02-22T20:52:42.190685+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41816,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.195716+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:42.195716+0000","last_peered":"2026-02-22T20:52:42.195716+0000","last_clean":"2026-02-22T20:52:42.195716+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:42.195716+0000","last_undegraded":"2026-02-22T20:52:42.195716+0000","last_fullsized":"2026-02-22T20:52:42.195716+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10118,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190735+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:42.190735+0000","last_peered":"2026-02-22T20:52:42.190735+0000","last_clean":"2026-02-22T20:52:42.190735+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:42.190735+0000","last_undegraded":"2026-02-22T20:52:42.190735+0000","last_fullsized":"2026-02-22T20:52:42.190735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40757,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190621+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:42.190621+0000","last_peered":"2026-02-22T20:52:42.190621+0000","last_clean":"2026-02-22T20:52:42.190621+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:42.190621+0000","last_undegraded":"2026-02-22T20:52:42.190621+0000","last_fullsized":"2026-02-22T20:52:42.190621+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15970,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189827+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:42.189827+0000","last_peered":"2026-02-22T20:52:42.189827+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:42.189827+0000","last_undegraded":"2026-02-22T20:52:42.189827+0000","last_fullsized":"2026-02-22T20:52:42.189827+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14814,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.197002+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:42.197002+0000","last_peered":"2026-02-22T20:52:42.197002+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:42.197002+0000","last_undegraded":"2026-02-22T20:52:42.197002+0000","last_fullsized":"2026-02-22T20:52:42.197002+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":68108,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46013,"num_bytes_recovered":214184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":910082048,"data_stored":1032151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051601,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182419,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460596,"kb_used_data":199012,"kb_used_omap":23,"kb_used_meta":261544,"kb_avail":93911244,"statfs":{"total":96636764160,"available":96165113856,"internally_reserved":0,"allocated":203788288,"data_stored":369868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":267821796},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482492,"kb_used_data":202860,"kb_used_omap":23,"kb_used_meta":279592,"kb_avail":93889348,"statfs":{"total":96636764160,"available":96142692352,"internally_reserved":0,"allocated":207728640,"data_stored":370830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":286302948},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207773696,"data_stored":249407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207773696,"data_stored":249407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326496256,"data_stored":354792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":168038400,"data_stored":178545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:45.252 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:45.252 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 15702, 'reported_epoch': 39, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:42.189892+0000', 'last_change': '2026-02-22T20:52:31.180859+0000', 'last_active': '2026-02-22T20:52:42.189892+0000', 'last_peered': '2026-02-22T20:52:42.189892+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:42.189892+0000', 'last_undegraded': '2026-02-22T20:52:42.189892+0000', 'last_fullsized': '2026-02-22T20:52:42.189892+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16792, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1810, 'num_bytes_recovered': 1810, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:45.252 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:45.252 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 15560, 'reported_epoch': 39, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:52:42.287698+0000', 'last_change': '2026-02-22T20:52:42.197046+0000', 'last_active': '2026-02-22T20:52:42.287698+0000', 'last_peered': '2026-02-22T20:52:42.287698+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:52:42.287698+0000', 'last_undegraded': '2026-02-22T20:52:42.287698+0000', 'last_fullsized': '2026-02-22T20:52:42.287698+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17126, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1720, 'num_bytes_recovered': 17003, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:45.252 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:45.252 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 15970, 'reported_epoch': 39, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:42.189827+0000', 'last_change': '2026-02-22T20:52:20.160334+0000', 'last_active': '2026-02-22T20:52:42.189827+0000', 'last_peered': '2026-02-22T20:52:42.189827+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:42.189827+0000', 'last_undegraded': '2026-02-22T20:52:42.189827+0000', 'last_fullsized': '2026-02-22T20:52:42.189827+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16712, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1883, 'num_bytes_recovered': 1883, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:45.252 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:45.252 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 14814, 'reported_epoch': 39, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:42.197002+0000', 'last_change': '2026-02-22T20:52:42.196267+0000', 'last_active': '2026-02-22T20:52:42.197002+0000', 'last_peered': '2026-02-22T20:52:42.197002+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:42.197002+0000', 'last_undegraded': '2026-02-22T20:52:42.197002+0000', 'last_fullsized': '2026-02-22T20:52:42.197002+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1501, 'num_bytes_recovered': 1501, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:45.253 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:45.482 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:45.482 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1182,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":39,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":4},{"state_name":"active+remapped+backfill_wait","count":3},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":965722112,"bytes_avail":192307806208,"bytes_total":193273528320,"misplaced_objects":68108,"misplaced_total":163508,"misplaced_ratio":0.4165423098563984,"recovering_objects_per_sec":1354,"recovering_bytes_per_sec":8995,"recovering_keys_per_sec":0,"num_objects_recovered":10837,"num_bytes_recovered":71969,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:52:45.497 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:45.498 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:45.689 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:45.690 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:45.703 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":626,"stamp":"2026-02-22T20:52:45.121445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":68108,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46013,"num_bytes_recovered":214184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":943088,"kb_used_data":401872,"kb_used_omap":46,"kb_used_meta":541137,"kb_avail":187800592,"statfs":{"total":193273528320,"available":192307806208,"internally_reserved":0,"allocated":411516928,"data_stored":740698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47672,"internal_metadata":554124744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21674,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10837,"num_bytes_recovered":71969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000875"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15702,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189892+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:42.189892+0000","last_peered":"2026-02-22T20:52:42.189892+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:42.189892+0000","last_undegraded":"2026-02-22T20:52:42.189892+0000","last_fullsized":"2026-02-22T20:52:42.189892+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15560,"reported_epoch":39,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:42.287698+0000","last_change":"2026-02-22T20:52:42.197046+0000","last_active":"2026-02-22T20:52:42.287698+0000","last_peered":"2026-02-22T20:52:42.287698+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:42.287698+0000","last_undegraded":"2026-02-22T20:52:42.287698+0000","last_fullsized":"2026-02-22T20:52:42.287698+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17126,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1720,"num_bytes_recovered":17003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40497,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190685+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:42.190685+0000","last_peered":"2026-02-22T20:52:42.190685+0000","last_clean":"2026-02-22T20:52:42.190685+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:42.190685+0000","last_undegraded":"2026-02-22T20:52:42.190685+0000","last_fullsized":"2026-02-22T20:52:42.190685+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41816,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.195716+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:42.195716+0000","last_peered":"2026-02-22T20:52:42.195716+0000","last_clean":"2026-02-22T20:52:42.195716+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:42.195716+0000","last_undegraded":"2026-02-22T20:52:42.195716+0000","last_fullsized":"2026-02-22T20:52:42.195716+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10118,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190735+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:42.190735+0000","last_peered":"2026-02-22T20:52:42.190735+0000","last_clean":"2026-02-22T20:52:42.190735+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:42.190735+0000","last_undegraded":"2026-02-22T20:52:42.190735+0000","last_fullsized":"2026-02-22T20:52:42.190735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40757,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190621+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:42.190621+0000","last_peered":"2026-02-22T20:52:42.190621+0000","last_clean":"2026-02-22T20:52:42.190621+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:42.190621+0000","last_undegraded":"2026-02-22T20:52:42.190621+0000","last_fullsized":"2026-02-22T20:52:42.190621+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15970,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189827+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:42.189827+0000","last_peered":"2026-02-22T20:52:42.189827+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:42.189827+0000","last_undegraded":"2026-02-22T20:52:42.189827+0000","last_fullsized":"2026-02-22T20:52:42.189827+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14814,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.197002+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:42.197002+0000","last_peered":"2026-02-22T20:52:42.197002+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:42.197002+0000","last_undegraded":"2026-02-22T20:52:42.197002+0000","last_fullsized":"2026-02-22T20:52:42.197002+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":68108,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46013,"num_bytes_recovered":214184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":910082048,"data_stored":1032151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051601,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182419,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460596,"kb_used_data":199012,"kb_used_omap":23,"kb_used_meta":261544,"kb_avail":93911244,"statfs":{"total":96636764160,"available":96165113856,"internally_reserved":0,"allocated":203788288,"data_stored":369868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":267821796},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482492,"kb_used_data":202860,"kb_used_omap":23,"kb_used_meta":279592,"kb_avail":93889348,"statfs":{"total":96636764160,"available":96142692352,"internally_reserved":0,"allocated":207728640,"data_stored":370830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":286302948},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207773696,"data_stored":249407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207773696,"data_stored":249407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326496256,"data_stored":354792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":168038400,"data_stored":178545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:48.706 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:48.897 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:48.897 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:48.912 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":627,"stamp":"2026-02-22T20:52:47.121663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":68108,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46013,"num_bytes_recovered":214184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":943088,"kb_used_data":401872,"kb_used_omap":46,"kb_used_meta":541137,"kb_avail":187800592,"statfs":{"total":193273528320,"available":192307806208,"internally_reserved":0,"allocated":411516928,"data_stored":740698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47672,"internal_metadata":554124744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10740,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5370,"num_bytes_recovered":51219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000883"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15702,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189892+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:42.189892+0000","last_peered":"2026-02-22T20:52:42.189892+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:42.189892+0000","last_undegraded":"2026-02-22T20:52:42.189892+0000","last_fullsized":"2026-02-22T20:52:42.189892+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15560,"reported_epoch":39,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:42.287698+0000","last_change":"2026-02-22T20:52:42.197046+0000","last_active":"2026-02-22T20:52:42.287698+0000","last_peered":"2026-02-22T20:52:42.287698+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:42.287698+0000","last_undegraded":"2026-02-22T20:52:42.287698+0000","last_fullsized":"2026-02-22T20:52:42.287698+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17126,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1720,"num_bytes_recovered":17003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40497,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190685+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:42.190685+0000","last_peered":"2026-02-22T20:52:42.190685+0000","last_clean":"2026-02-22T20:52:42.190685+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:42.190685+0000","last_undegraded":"2026-02-22T20:52:42.190685+0000","last_fullsized":"2026-02-22T20:52:42.190685+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41816,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.195716+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:42.195716+0000","last_peered":"2026-02-22T20:52:42.195716+0000","last_clean":"2026-02-22T20:52:42.195716+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:42.195716+0000","last_undegraded":"2026-02-22T20:52:42.195716+0000","last_fullsized":"2026-02-22T20:52:42.195716+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10118,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190735+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:42.190735+0000","last_peered":"2026-02-22T20:52:42.190735+0000","last_clean":"2026-02-22T20:52:42.190735+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:42.190735+0000","last_undegraded":"2026-02-22T20:52:42.190735+0000","last_fullsized":"2026-02-22T20:52:42.190735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40757,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190621+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:42.190621+0000","last_peered":"2026-02-22T20:52:42.190621+0000","last_clean":"2026-02-22T20:52:42.190621+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:42.190621+0000","last_undegraded":"2026-02-22T20:52:42.190621+0000","last_fullsized":"2026-02-22T20:52:42.190621+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15970,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189827+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:42.189827+0000","last_peered":"2026-02-22T20:52:42.189827+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:42.189827+0000","last_undegraded":"2026-02-22T20:52:42.189827+0000","last_fullsized":"2026-02-22T20:52:42.189827+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14814,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.197002+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:42.197002+0000","last_peered":"2026-02-22T20:52:42.197002+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:42.197002+0000","last_undegraded":"2026-02-22T20:52:42.197002+0000","last_fullsized":"2026-02-22T20:52:42.197002+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":68108,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46013,"num_bytes_recovered":214184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":910082048,"data_stored":1032151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051601,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182419,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460596,"kb_used_data":199012,"kb_used_omap":23,"kb_used_meta":261544,"kb_avail":93911244,"statfs":{"total":96636764160,"available":96165113856,"internally_reserved":0,"allocated":203788288,"data_stored":369868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":267821796},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482492,"kb_used_data":202860,"kb_used_omap":23,"kb_used_meta":279592,"kb_avail":93889348,"statfs":{"total":96636764160,"available":96142692352,"internally_reserved":0,"allocated":207728640,"data_stored":370830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":286302948},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207773696,"data_stored":249407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207773696,"data_stored":249407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326496256,"data_stored":354792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":168038400,"data_stored":178545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:48.914 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:49.106 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:49.107 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:49.119 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":627,"stamp":"2026-02-22T20:52:47.121663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":68108,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46013,"num_bytes_recovered":214184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":943088,"kb_used_data":401872,"kb_used_omap":46,"kb_used_meta":541137,"kb_avail":187800592,"statfs":{"total":193273528320,"available":192307806208,"internally_reserved":0,"allocated":411516928,"data_stored":740698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47672,"internal_metadata":554124744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10740,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5370,"num_bytes_recovered":51219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000883"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15702,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189892+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:42.189892+0000","last_peered":"2026-02-22T20:52:42.189892+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:42.189892+0000","last_undegraded":"2026-02-22T20:52:42.189892+0000","last_fullsized":"2026-02-22T20:52:42.189892+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":15560,"reported_epoch":39,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:42.287698+0000","last_change":"2026-02-22T20:52:42.197046+0000","last_active":"2026-02-22T20:52:42.287698+0000","last_peered":"2026-02-22T20:52:42.287698+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:42.287698+0000","last_undegraded":"2026-02-22T20:52:42.287698+0000","last_fullsized":"2026-02-22T20:52:42.287698+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17126,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1720,"num_bytes_recovered":17003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40497,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190685+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:42.190685+0000","last_peered":"2026-02-22T20:52:42.190685+0000","last_clean":"2026-02-22T20:52:42.190685+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:42.190685+0000","last_undegraded":"2026-02-22T20:52:42.190685+0000","last_fullsized":"2026-02-22T20:52:42.190685+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41816,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.195716+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:42.195716+0000","last_peered":"2026-02-22T20:52:42.195716+0000","last_clean":"2026-02-22T20:52:42.195716+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:42.195716+0000","last_undegraded":"2026-02-22T20:52:42.195716+0000","last_fullsized":"2026-02-22T20:52:42.195716+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10118,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190735+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:42.190735+0000","last_peered":"2026-02-22T20:52:42.190735+0000","last_clean":"2026-02-22T20:52:42.190735+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:42.190735+0000","last_undegraded":"2026-02-22T20:52:42.190735+0000","last_fullsized":"2026-02-22T20:52:42.190735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40757,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190621+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:42.190621+0000","last_peered":"2026-02-22T20:52:42.190621+0000","last_clean":"2026-02-22T20:52:42.190621+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:42.190621+0000","last_undegraded":"2026-02-22T20:52:42.190621+0000","last_fullsized":"2026-02-22T20:52:42.190621+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15970,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189827+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:42.189827+0000","last_peered":"2026-02-22T20:52:42.189827+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:42.189827+0000","last_undegraded":"2026-02-22T20:52:42.189827+0000","last_fullsized":"2026-02-22T20:52:42.189827+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14814,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.197002+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:42.197002+0000","last_peered":"2026-02-22T20:52:42.197002+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:42.197002+0000","last_undegraded":"2026-02-22T20:52:42.197002+0000","last_fullsized":"2026-02-22T20:52:42.197002+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":68108,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46013,"num_bytes_recovered":214184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":910082048,"data_stored":1032151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051601,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182419,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460596,"kb_used_data":199012,"kb_used_omap":23,"kb_used_meta":261544,"kb_avail":93911244,"statfs":{"total":96636764160,"available":96165113856,"internally_reserved":0,"allocated":203788288,"data_stored":369868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":267821796},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482492,"kb_used_data":202860,"kb_used_omap":23,"kb_used_meta":279592,"kb_avail":93889348,"statfs":{"total":96636764160,"available":96142692352,"internally_reserved":0,"allocated":207728640,"data_stored":370830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":286302948},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207773696,"data_stored":249407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207773696,"data_stored":249407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326496256,"data_stored":354792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":168038400,"data_stored":178545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:49.121 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:49.121 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 15702, 'reported_epoch': 39, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:42.189892+0000', 'last_change': '2026-02-22T20:52:31.180859+0000', 'last_active': '2026-02-22T20:52:42.189892+0000', 'last_peered': '2026-02-22T20:52:42.189892+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:42.189892+0000', 'last_undegraded': '2026-02-22T20:52:42.189892+0000', 'last_fullsized': '2026-02-22T20:52:42.189892+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16792, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1810, 'num_bytes_recovered': 1810, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:49.121 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:49.121 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 15560, 'reported_epoch': 39, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:52:42.287698+0000', 'last_change': '2026-02-22T20:52:42.197046+0000', 'last_active': '2026-02-22T20:52:42.287698+0000', 'last_peered': '2026-02-22T20:52:42.287698+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:52:42.287698+0000', 'last_undegraded': '2026-02-22T20:52:42.287698+0000', 'last_fullsized': '2026-02-22T20:52:42.287698+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17126, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1720, 'num_bytes_recovered': 17003, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:49.122 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:49.122 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 15970, 'reported_epoch': 39, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:42.189827+0000', 'last_change': '2026-02-22T20:52:20.160334+0000', 'last_active': '2026-02-22T20:52:42.189827+0000', 'last_peered': '2026-02-22T20:52:42.189827+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:42.189827+0000', 'last_undegraded': '2026-02-22T20:52:42.189827+0000', 'last_fullsized': '2026-02-22T20:52:42.189827+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16712, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1883, 'num_bytes_recovered': 1883, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:49.122 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:49.122 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 14814, 'reported_epoch': 39, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:42.197002+0000', 'last_change': '2026-02-22T20:52:42.196267+0000', 'last_active': '2026-02-22T20:52:42.197002+0000', 'last_peered': '2026-02-22T20:52:42.197002+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:42.197002+0000', 'last_undegraded': '2026-02-22T20:52:42.197002+0000', 'last_fullsized': '2026-02-22T20:52:42.197002+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1501, 'num_bytes_recovered': 1501, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:49.122 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:49.354 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:49.354 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1186,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":39,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":4},{"state_name":"active+remapped+backfill_wait","count":3},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":965722112,"bytes_avail":192307806208,"bytes_total":193273528320,"misplaced_objects":57156,"misplaced_total":163508,"misplaced_ratio":0.34956087775521688,"recovering_objects_per_sec":1355,"recovering_bytes_per_sec":12816,"recovering_keys_per_sec":0,"num_objects_recovered":10846,"num_bytes_recovered":102544,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:52:49.369 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:49.370 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:49.560 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:49.561 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:49.575 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":628,"stamp":"2026-02-22T20:52:49.121875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":57156,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51489,"num_bytes_recovered":265509,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":943088,"kb_used_data":401872,"kb_used_omap":46,"kb_used_meta":541137,"kb_avail":187800592,"statfs":{"total":193273528320,"available":192307806208,"internally_reserved":0,"allocated":411516928,"data_stored":740698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47672,"internal_metadata":554124744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21692,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10846,"num_bytes_recovered":102544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000875"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15702,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189892+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:42.189892+0000","last_peered":"2026-02-22T20:52:42.189892+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:42.189892+0000","last_undegraded":"2026-02-22T20:52:42.189892+0000","last_fullsized":"2026-02-22T20:52:42.189892+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":31988,"reported_epoch":39,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:47.288121+0000","last_change":"2026-02-22T20:52:42.197046+0000","last_active":"2026-02-22T20:52:47.288121+0000","last_peered":"2026-02-22T20:52:47.288121+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:47.288121+0000","last_undegraded":"2026-02-22T20:52:47.288121+0000","last_fullsized":"2026-02-22T20:52:47.288121+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6174,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7196,"num_bytes_recovered":68328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40497,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190685+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:42.190685+0000","last_peered":"2026-02-22T20:52:42.190685+0000","last_clean":"2026-02-22T20:52:42.190685+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:42.190685+0000","last_undegraded":"2026-02-22T20:52:42.190685+0000","last_fullsized":"2026-02-22T20:52:42.190685+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41816,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.195716+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:42.195716+0000","last_peered":"2026-02-22T20:52:42.195716+0000","last_clean":"2026-02-22T20:52:42.195716+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:42.195716+0000","last_undegraded":"2026-02-22T20:52:42.195716+0000","last_fullsized":"2026-02-22T20:52:42.195716+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10118,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190735+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:42.190735+0000","last_peered":"2026-02-22T20:52:42.190735+0000","last_clean":"2026-02-22T20:52:42.190735+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:42.190735+0000","last_undegraded":"2026-02-22T20:52:42.190735+0000","last_fullsized":"2026-02-22T20:52:42.190735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40757,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190621+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:42.190621+0000","last_peered":"2026-02-22T20:52:42.190621+0000","last_clean":"2026-02-22T20:52:42.190621+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:42.190621+0000","last_undegraded":"2026-02-22T20:52:42.190621+0000","last_fullsized":"2026-02-22T20:52:42.190621+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15970,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189827+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:42.189827+0000","last_peered":"2026-02-22T20:52:42.189827+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:42.189827+0000","last_undegraded":"2026-02-22T20:52:42.189827+0000","last_fullsized":"2026-02-22T20:52:42.189827+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14814,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.197002+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:42.197002+0000","last_peered":"2026-02-22T20:52:42.197002+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:42.197002+0000","last_undegraded":"2026-02-22T20:52:42.197002+0000","last_fullsized":"2026-02-22T20:52:42.197002+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":57156,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51489,"num_bytes_recovered":265509,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":907612160,"data_stored":1016268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051602,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182420,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705905,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460596,"kb_used_data":199012,"kb_used_omap":23,"kb_used_meta":261544,"kb_avail":93911244,"statfs":{"total":96636764160,"available":96165113856,"internally_reserved":0,"allocated":203788288,"data_stored":369868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":267821796},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482492,"kb_used_data":202860,"kb_used_omap":23,"kb_used_meta":279592,"kb_avail":93889348,"statfs":{"total":96636764160,"available":96142692352,"internally_reserved":0,"allocated":207728640,"data_stored":370830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":286302948},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207773696,"data_stored":249407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207773696,"data_stored":249407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324026368,"data_stored":338909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":168038400,"data_stored":178545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:52.577 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:52.771 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:52.771 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:52.787 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":631,"stamp":"2026-02-22T20:52:51.200523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":57156,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51489,"num_bytes_recovered":265509,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1037368,"kb_used_data":442008,"kb_used_omap":46,"kb_used_meta":595281,"kb_avail":187706312,"statfs":{"total":193273528320,"available":192211263488,"internally_reserved":0,"allocated":452616192,"data_stored":781292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47672,"internal_metadata":609568200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-18504,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9251,"num_bytes_recovered":100949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.079306"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15702,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189892+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:42.189892+0000","last_peered":"2026-02-22T20:52:42.189892+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:42.189892+0000","last_undegraded":"2026-02-22T20:52:42.189892+0000","last_fullsized":"2026-02-22T20:52:42.189892+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":31988,"reported_epoch":39,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:47.288121+0000","last_change":"2026-02-22T20:52:42.197046+0000","last_active":"2026-02-22T20:52:47.288121+0000","last_peered":"2026-02-22T20:52:47.288121+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:47.288121+0000","last_undegraded":"2026-02-22T20:52:47.288121+0000","last_fullsized":"2026-02-22T20:52:47.288121+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6174,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7196,"num_bytes_recovered":68328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40497,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190685+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:42.190685+0000","last_peered":"2026-02-22T20:52:42.190685+0000","last_clean":"2026-02-22T20:52:42.190685+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:42.190685+0000","last_undegraded":"2026-02-22T20:52:42.190685+0000","last_fullsized":"2026-02-22T20:52:42.190685+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41816,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.195716+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:42.195716+0000","last_peered":"2026-02-22T20:52:42.195716+0000","last_clean":"2026-02-22T20:52:42.195716+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:42.195716+0000","last_undegraded":"2026-02-22T20:52:42.195716+0000","last_fullsized":"2026-02-22T20:52:42.195716+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10118,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190735+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:42.190735+0000","last_peered":"2026-02-22T20:52:42.190735+0000","last_clean":"2026-02-22T20:52:42.190735+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:42.190735+0000","last_undegraded":"2026-02-22T20:52:42.190735+0000","last_fullsized":"2026-02-22T20:52:42.190735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40757,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190621+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:42.190621+0000","last_peered":"2026-02-22T20:52:42.190621+0000","last_clean":"2026-02-22T20:52:42.190621+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:42.190621+0000","last_undegraded":"2026-02-22T20:52:42.190621+0000","last_fullsized":"2026-02-22T20:52:42.190621+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15970,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189827+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:42.189827+0000","last_peered":"2026-02-22T20:52:42.189827+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:42.189827+0000","last_undegraded":"2026-02-22T20:52:42.189827+0000","last_fullsized":"2026-02-22T20:52:42.189827+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14814,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.197002+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:42.197002+0000","last_peered":"2026-02-22T20:52:42.197002+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:42.197002+0000","last_undegraded":"2026-02-22T20:52:42.197002+0000","last_fullsized":"2026-02-22T20:52:42.197002+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":57156,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51489,"num_bytes_recovered":265509,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":952385536,"data_stored":1057759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051602,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182420,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518844,"kb_used_data":221164,"kb_used_omap":23,"kb_used_meta":297640,"kb_avail":93852996,"statfs":{"total":96636764160,"available":96105467904,"internally_reserved":0,"allocated":226471936,"data_stored":390686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":304784100},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518524,"kb_used_data":220844,"kb_used_omap":23,"kb_used_meta":297640,"kb_avail":93853316,"statfs":{"total":96636764160,"available":96105795584,"internally_reserved":0,"allocated":226144256,"data_stored":390606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":304784100},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":230162432,"data_stored":270153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230158336,"data_stored":270152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324026368,"data_stored":338909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":168038400,"data_stored":178545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:52.788 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:52.980 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:52.980 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:52.994 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":631,"stamp":"2026-02-22T20:52:51.200523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":57156,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51489,"num_bytes_recovered":265509,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1037368,"kb_used_data":442008,"kb_used_omap":46,"kb_used_meta":595281,"kb_avail":187706312,"statfs":{"total":193273528320,"available":192211263488,"internally_reserved":0,"allocated":452616192,"data_stored":781292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47672,"internal_metadata":609568200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-18504,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9251,"num_bytes_recovered":100949,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.079306"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":15702,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189892+0000","last_change":"2026-02-22T20:52:31.180859+0000","last_active":"2026-02-22T20:52:42.189892+0000","last_peered":"2026-02-22T20:52:42.189892+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:42.189892+0000","last_undegraded":"2026-02-22T20:52:42.189892+0000","last_fullsized":"2026-02-22T20:52:42.189892+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16792,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1810,"num_bytes_recovered":1810,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":31988,"reported_epoch":39,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:47.288121+0000","last_change":"2026-02-22T20:52:42.197046+0000","last_active":"2026-02-22T20:52:47.288121+0000","last_peered":"2026-02-22T20:52:47.288121+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:47.288121+0000","last_undegraded":"2026-02-22T20:52:47.288121+0000","last_fullsized":"2026-02-22T20:52:47.288121+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6174,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7196,"num_bytes_recovered":68328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40497,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190685+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:42.190685+0000","last_peered":"2026-02-22T20:52:42.190685+0000","last_clean":"2026-02-22T20:52:42.190685+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:42.190685+0000","last_undegraded":"2026-02-22T20:52:42.190685+0000","last_fullsized":"2026-02-22T20:52:42.190685+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41816,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.195716+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:42.195716+0000","last_peered":"2026-02-22T20:52:42.195716+0000","last_clean":"2026-02-22T20:52:42.195716+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:42.195716+0000","last_undegraded":"2026-02-22T20:52:42.195716+0000","last_fullsized":"2026-02-22T20:52:42.195716+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10118,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190735+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:42.190735+0000","last_peered":"2026-02-22T20:52:42.190735+0000","last_clean":"2026-02-22T20:52:42.190735+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:42.190735+0000","last_undegraded":"2026-02-22T20:52:42.190735+0000","last_fullsized":"2026-02-22T20:52:42.190735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40757,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190621+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:42.190621+0000","last_peered":"2026-02-22T20:52:42.190621+0000","last_clean":"2026-02-22T20:52:42.190621+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:42.190621+0000","last_undegraded":"2026-02-22T20:52:42.190621+0000","last_fullsized":"2026-02-22T20:52:42.190621+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":15970,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.189827+0000","last_change":"2026-02-22T20:52:20.160334+0000","last_active":"2026-02-22T20:52:42.189827+0000","last_peered":"2026-02-22T20:52:42.189827+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:42.189827+0000","last_undegraded":"2026-02-22T20:52:42.189827+0000","last_fullsized":"2026-02-22T20:52:42.189827+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16712,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1883,"num_bytes_recovered":1883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14814,"reported_epoch":39,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:42.197002+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:42.197002+0000","last_peered":"2026-02-22T20:52:42.197002+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:42.197002+0000","last_undegraded":"2026-02-22T20:52:42.197002+0000","last_fullsized":"2026-02-22T20:52:42.197002+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":57156,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51489,"num_bytes_recovered":265509,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":952385536,"data_stored":1057759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051602,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182420,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518844,"kb_used_data":221164,"kb_used_omap":23,"kb_used_meta":297640,"kb_avail":93852996,"statfs":{"total":96636764160,"available":96105467904,"internally_reserved":0,"allocated":226471936,"data_stored":390686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":304784100},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518524,"kb_used_data":220844,"kb_used_omap":23,"kb_used_meta":297640,"kb_avail":93853316,"statfs":{"total":96636764160,"available":96105795584,"internally_reserved":0,"allocated":226144256,"data_stored":390606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":304784100},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":230162432,"data_stored":270153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230158336,"data_stored":270152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324026368,"data_stored":338909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":168038400,"data_stored":178545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:52.994 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:52.995 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 15702, 'reported_epoch': 39, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:42.189892+0000', 'last_change': '2026-02-22T20:52:31.180859+0000', 'last_active': '2026-02-22T20:52:42.189892+0000', 'last_peered': '2026-02-22T20:52:42.189892+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:42.189892+0000', 'last_undegraded': '2026-02-22T20:52:42.189892+0000', 'last_fullsized': '2026-02-22T20:52:42.189892+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16792, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1810, 'num_bytes_recovered': 1810, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:52.995 INFO:tasks.osd_recovery.ceph_manager:PG 1.6 is not active+clean 2026-02-22T20:52:52.995 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.6', 'version': "17'10316", 'reported_seq': 31988, 'reported_epoch': 39, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:52:47.288121+0000', 'last_change': '2026-02-22T20:52:42.197046+0000', 'last_active': '2026-02-22T20:52:47.288121+0000', 'last_peered': '2026-02-22T20:52:47.288121+0000', 'last_clean': '2026-02-22T20:50:30.395451+0000', 'last_became_active': '2026-02-22T20:51:26.182390+0000', 'last_became_peered': '2026-02-22T20:51:26.182390+0000', 'last_unstale': '2026-02-22T20:52:47.288121+0000', 'last_undegraded': '2026-02-22T20:52:47.288121+0000', 'last_fullsized': '2026-02-22T20:52:47.288121+0000', 'mapping_epoch': 30, 'log_start': "17'300", 'ondisk_log_start': "17'300", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000779827, 'stat_sum': {'num_bytes': 71448, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 6174, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10316, 'num_write_kb': 10372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7196, 'num_bytes_recovered': 68328, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10316}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:52.995 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:52.995 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 15970, 'reported_epoch': 39, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:42.189827+0000', 'last_change': '2026-02-22T20:52:20.160334+0000', 'last_active': '2026-02-22T20:52:42.189827+0000', 'last_peered': '2026-02-22T20:52:42.189827+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:42.189827+0000', 'last_undegraded': '2026-02-22T20:52:42.189827+0000', 'last_fullsized': '2026-02-22T20:52:42.189827+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 16712, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1883, 'num_bytes_recovered': 1883, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:52.995 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:52.995 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 14814, 'reported_epoch': 39, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:42.197002+0000', 'last_change': '2026-02-22T20:52:42.196267+0000', 'last_active': '2026-02-22T20:52:42.197002+0000', 'last_peered': '2026-02-22T20:52:42.197002+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:42.197002+0000', 'last_undegraded': '2026-02-22T20:52:42.197002+0000', 'last_fullsized': '2026-02-22T20:52:42.197002+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1501, 'num_bytes_recovered': 1501, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:52.995 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:53.230 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:53.230 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1189,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":41,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":4},{"state_name":"active+remapped+backfill_wait","count":2},{"state_name":"active+remapped+backfilling","count":2}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":1062264832,"bytes_avail":192211263488,"bytes_total":193273528320,"misplaced_objects":52412,"misplaced_total":163508,"misplaced_ratio":0.32054700687428139,"recovering_objects_per_sec":980,"recovering_bytes_per_sec":6711,"recovering_keys_per_sec":0,"num_objects_recovered":7848,"num_bytes_recovered":53697,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:52:53.246 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:53.246 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:53.434 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:53.434 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:53.448 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":632,"stamp":"2026-02-22T20:52:53.122332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":52412,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53861,"num_bytes_recovered":267881,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1037368,"kb_used_data":442008,"kb_used_omap":46,"kb_used_meta":595281,"kb_avail":187706312,"statfs":{"total":193273528320,"available":192211263488,"internally_reserved":0,"allocated":452616192,"data_stored":781292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47672,"internal_metadata":609568200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-15696,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7848,"num_bytes_recovered":53697,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000886"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":19209,"reported_epoch":41,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:52.288895+0000","last_change":"2026-02-22T20:52:51.210008+0000","last_active":"2026-02-22T20:52:52.288895+0000","last_peered":"2026-02-22T20:52:52.288895+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:52.288895+0000","last_undegraded":"2026-02-22T20:52:52.288895+0000","last_fullsized":"2026-02-22T20:52:52.288895+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14458,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2977,"num_bytes_recovered":2977,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":31988,"reported_epoch":39,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:47.288121+0000","last_change":"2026-02-22T20:52:42.197046+0000","last_active":"2026-02-22T20:52:47.288121+0000","last_peered":"2026-02-22T20:52:47.288121+0000","last_clean":"2026-02-22T20:50:30.395451+0000","last_became_active":"2026-02-22T20:51:26.182390+0000","last_became_peered":"2026-02-22T20:51:26.182390+0000","last_unstale":"2026-02-22T20:52:47.288121+0000","last_undegraded":"2026-02-22T20:52:47.288121+0000","last_fullsized":"2026-02-22T20:52:47.288121+0000","mapping_epoch":30,"log_start":"17'300","ondisk_log_start":"17'300","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6174,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7196,"num_bytes_recovered":68328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10316}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40497,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190685+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:42.190685+0000","last_peered":"2026-02-22T20:52:42.190685+0000","last_clean":"2026-02-22T20:52:42.190685+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:42.190685+0000","last_undegraded":"2026-02-22T20:52:42.190685+0000","last_fullsized":"2026-02-22T20:52:42.190685+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41816,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.195716+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:42.195716+0000","last_peered":"2026-02-22T20:52:42.195716+0000","last_clean":"2026-02-22T20:52:42.195716+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:42.195716+0000","last_undegraded":"2026-02-22T20:52:42.195716+0000","last_fullsized":"2026-02-22T20:52:42.195716+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10118,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190735+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:42.190735+0000","last_peered":"2026-02-22T20:52:42.190735+0000","last_clean":"2026-02-22T20:52:42.190735+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:42.190735+0000","last_undegraded":"2026-02-22T20:52:42.190735+0000","last_fullsized":"2026-02-22T20:52:42.190735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40757,"reported_epoch":39,"state":"active+clean","last_fresh":"2026-02-22T20:52:42.190621+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:42.190621+0000","last_peered":"2026-02-22T20:52:42.190621+0000","last_clean":"2026-02-22T20:52:42.190621+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:42.190621+0000","last_undegraded":"2026-02-22T20:52:42.190621+0000","last_fullsized":"2026-02-22T20:52:42.190621+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":19593,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.210026+0000","last_change":"2026-02-22T20:52:51.209471+0000","last_active":"2026-02-22T20:52:51.210026+0000","last_peered":"2026-02-22T20:52:51.210026+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:51.210026+0000","last_undegraded":"2026-02-22T20:52:51.210026+0000","last_fullsized":"2026-02-22T20:52:51.210026+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14302,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3088,"num_bytes_recovered":3088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14818,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.201275+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:51.201275+0000","last_peered":"2026-02-22T20:52:51.201275+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:51.201275+0000","last_undegraded":"2026-02-22T20:52:51.201275+0000","last_fullsized":"2026-02-22T20:52:51.201275+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":52412,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53861,"num_bytes_recovered":267881,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":949673984,"data_stored":1041817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80153,"ondisk_log_size":80153,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051602,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182421,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705906,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518844,"kb_used_data":221164,"kb_used_omap":23,"kb_used_meta":297640,"kb_avail":93852996,"statfs":{"total":96636764160,"available":96105467904,"internally_reserved":0,"allocated":226471936,"data_stored":390686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":304784100},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518524,"kb_used_data":220844,"kb_used_omap":23,"kb_used_meta":297640,"kb_avail":93853316,"statfs":{"total":96636764160,"available":96105795584,"internally_reserved":0,"allocated":226144256,"data_stored":390606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":304784100},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":230162432,"data_stored":270153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230158336,"data_stored":270152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321314816,"data_stored":322967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":168038400,"data_stored":178545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:56.450 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:56.642 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:56.643 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:56.657 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":633,"stamp":"2026-02-22T20:52:55.122546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":46238,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56947,"num_bytes_recovered":270967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80137,"ondisk_log_size":80137,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1153356,"kb_used_data":485804,"kb_used_omap":49,"kb_used_meta":667470,"kb_avail":187590324,"statfs":{"total":193273528320,"available":192092491776,"internally_reserved":0,"allocated":497463296,"data_stored":805383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":50276,"internal_metadata":683490204},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21870,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10934,"num_bytes_recovered":56783,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000883"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":19209,"reported_epoch":41,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:52.288895+0000","last_change":"2026-02-22T20:52:51.210008+0000","last_active":"2026-02-22T20:52:52.288895+0000","last_peered":"2026-02-22T20:52:52.288895+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:52.288895+0000","last_undegraded":"2026-02-22T20:52:52.288895+0000","last_fullsized":"2026-02-22T20:52:52.288895+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14458,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2977,"num_bytes_recovered":2977,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41257,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.208446+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:52:51.208446+0000","last_peered":"2026-02-22T20:52:51.208446+0000","last_clean":"2026-02-22T20:52:51.208446+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:52:51.208446+0000","last_undegraded":"2026-02-22T20:52:51.208446+0000","last_fullsized":"2026-02-22T20:52:51.208446+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40501,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.202401+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:51.202401+0000","last_peered":"2026-02-22T20:52:51.202401+0000","last_clean":"2026-02-22T20:52:51.202401+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:51.202401+0000","last_undegraded":"2026-02-22T20:52:51.202401+0000","last_fullsized":"2026-02-22T20:52:51.202401+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41820,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.202356+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:51.202356+0000","last_peered":"2026-02-22T20:52:51.202356+0000","last_clean":"2026-02-22T20:52:51.202356+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:51.202356+0000","last_undegraded":"2026-02-22T20:52:51.202356+0000","last_fullsized":"2026-02-22T20:52:51.202356+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10122,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.203011+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:51.203011+0000","last_peered":"2026-02-22T20:52:51.203011+0000","last_clean":"2026-02-22T20:52:51.203011+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:51.203011+0000","last_undegraded":"2026-02-22T20:52:51.203011+0000","last_fullsized":"2026-02-22T20:52:51.203011+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40761,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.202351+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:51.202351+0000","last_peered":"2026-02-22T20:52:51.202351+0000","last_clean":"2026-02-22T20:52:51.202351+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:51.202351+0000","last_undegraded":"2026-02-22T20:52:51.202351+0000","last_fullsized":"2026-02-22T20:52:51.202351+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":19593,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.210026+0000","last_change":"2026-02-22T20:52:51.209471+0000","last_active":"2026-02-22T20:52:51.210026+0000","last_peered":"2026-02-22T20:52:51.210026+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:51.210026+0000","last_undegraded":"2026-02-22T20:52:51.210026+0000","last_fullsized":"2026-02-22T20:52:51.210026+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14302,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3088,"num_bytes_recovered":3088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14818,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.201275+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:51.201275+0000","last_peered":"2026-02-22T20:52:51.201275+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:51.201275+0000","last_undegraded":"2026-02-22T20:52:51.201275+0000","last_fullsized":"2026-02-22T20:52:51.201275+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":46238,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56947,"num_bytes_recovered":270967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":951996416,"data_stored":981264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80137,"ondisk_log_size":80137,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051603,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182421,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576716,"kb_used_data":242940,"kb_used_omap":24,"kb_used_meta":333735,"kb_avail":93795124,"statfs":{"total":96636764160,"available":96046206976,"internally_reserved":0,"allocated":248770560,"data_stored":402701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":341745102},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576640,"kb_used_data":242864,"kb_used_omap":24,"kb_used_meta":333735,"kb_avail":93795200,"statfs":{"total":96636764160,"available":96046284800,"internally_reserved":0,"allocated":248692736,"data_stored":402682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":341745102},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":252473344,"data_stored":275600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252473344,"data_stored":275600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321314816,"data_stored":322967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":125734912,"data_stored":107097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:56.658 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:56.850 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:56.851 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:56.864 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":633,"stamp":"2026-02-22T20:52:55.122546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":46238,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56947,"num_bytes_recovered":270967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80137,"ondisk_log_size":80137,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1153356,"kb_used_data":485804,"kb_used_omap":49,"kb_used_meta":667470,"kb_avail":187590324,"statfs":{"total":193273528320,"available":192092491776,"internally_reserved":0,"allocated":497463296,"data_stored":805383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":50276,"internal_metadata":683490204},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21870,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10934,"num_bytes_recovered":56783,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000883"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":19209,"reported_epoch":41,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:52.288895+0000","last_change":"2026-02-22T20:52:51.210008+0000","last_active":"2026-02-22T20:52:52.288895+0000","last_peered":"2026-02-22T20:52:52.288895+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:52.288895+0000","last_undegraded":"2026-02-22T20:52:52.288895+0000","last_fullsized":"2026-02-22T20:52:52.288895+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14458,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2977,"num_bytes_recovered":2977,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41257,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.208446+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:52:51.208446+0000","last_peered":"2026-02-22T20:52:51.208446+0000","last_clean":"2026-02-22T20:52:51.208446+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:52:51.208446+0000","last_undegraded":"2026-02-22T20:52:51.208446+0000","last_fullsized":"2026-02-22T20:52:51.208446+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40501,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.202401+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:51.202401+0000","last_peered":"2026-02-22T20:52:51.202401+0000","last_clean":"2026-02-22T20:52:51.202401+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:51.202401+0000","last_undegraded":"2026-02-22T20:52:51.202401+0000","last_fullsized":"2026-02-22T20:52:51.202401+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41820,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.202356+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:51.202356+0000","last_peered":"2026-02-22T20:52:51.202356+0000","last_clean":"2026-02-22T20:52:51.202356+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:51.202356+0000","last_undegraded":"2026-02-22T20:52:51.202356+0000","last_fullsized":"2026-02-22T20:52:51.202356+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10122,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.203011+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:51.203011+0000","last_peered":"2026-02-22T20:52:51.203011+0000","last_clean":"2026-02-22T20:52:51.203011+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:51.203011+0000","last_undegraded":"2026-02-22T20:52:51.203011+0000","last_fullsized":"2026-02-22T20:52:51.203011+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40761,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.202351+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:51.202351+0000","last_peered":"2026-02-22T20:52:51.202351+0000","last_clean":"2026-02-22T20:52:51.202351+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:51.202351+0000","last_undegraded":"2026-02-22T20:52:51.202351+0000","last_fullsized":"2026-02-22T20:52:51.202351+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":19593,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.210026+0000","last_change":"2026-02-22T20:52:51.209471+0000","last_active":"2026-02-22T20:52:51.210026+0000","last_peered":"2026-02-22T20:52:51.210026+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:51.210026+0000","last_undegraded":"2026-02-22T20:52:51.210026+0000","last_fullsized":"2026-02-22T20:52:51.210026+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14302,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3088,"num_bytes_recovered":3088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14818,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.201275+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:51.201275+0000","last_peered":"2026-02-22T20:52:51.201275+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:51.201275+0000","last_undegraded":"2026-02-22T20:52:51.201275+0000","last_fullsized":"2026-02-22T20:52:51.201275+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":46238,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56947,"num_bytes_recovered":270967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":951996416,"data_stored":981264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80137,"ondisk_log_size":80137,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051603,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182421,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576716,"kb_used_data":242940,"kb_used_omap":24,"kb_used_meta":333735,"kb_avail":93795124,"statfs":{"total":96636764160,"available":96046206976,"internally_reserved":0,"allocated":248770560,"data_stored":402701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":341745102},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576640,"kb_used_data":242864,"kb_used_omap":24,"kb_used_meta":333735,"kb_avail":93795200,"statfs":{"total":96636764160,"available":96046284800,"internally_reserved":0,"allocated":248692736,"data_stored":402682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":341745102},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":252473344,"data_stored":275600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252473344,"data_stored":275600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321314816,"data_stored":322967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":125734912,"data_stored":107097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:52:56.865 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:52:56.865 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 19209, 'reported_epoch': 41, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:52:52.288895+0000', 'last_change': '2026-02-22T20:52:51.210008+0000', 'last_active': '2026-02-22T20:52:52.288895+0000', 'last_peered': '2026-02-22T20:52:52.288895+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:52.288895+0000', 'last_undegraded': '2026-02-22T20:52:52.288895+0000', 'last_fullsized': '2026-02-22T20:52:52.288895+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 14458, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2977, 'num_bytes_recovered': 2977, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:56.865 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:52:56.865 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 19593, 'reported_epoch': 41, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:51.210026+0000', 'last_change': '2026-02-22T20:52:51.209471+0000', 'last_active': '2026-02-22T20:52:51.210026+0000', 'last_peered': '2026-02-22T20:52:51.210026+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:51.210026+0000', 'last_undegraded': '2026-02-22T20:52:51.210026+0000', 'last_fullsized': '2026-02-22T20:52:51.210026+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 14302, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3088, 'num_bytes_recovered': 3088, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:56.865 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:52:56.865 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 14818, 'reported_epoch': 41, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:51.201275+0000', 'last_change': '2026-02-22T20:52:42.196267+0000', 'last_active': '2026-02-22T20:52:51.201275+0000', 'last_peered': '2026-02-22T20:52:51.201275+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:51.201275+0000', 'last_undegraded': '2026-02-22T20:52:51.201275+0000', 'last_fullsized': '2026-02-22T20:52:51.201275+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1501, 'num_bytes_recovered': 1501, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:52:56.865 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:52:57.098 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:57.098 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1193,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":41,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":5},{"state_name":"active+remapped+backfill_wait","count":2},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":1181036544,"bytes_avail":192092491776,"bytes_total":193273528320,"misplaced_objects":46238,"misplaced_total":163508,"misplaced_ratio":0.2827873865498936,"recovering_objects_per_sec":1366,"recovering_bytes_per_sec":7097,"recovering_keys_per_sec":0,"num_objects_recovered":10934,"num_bytes_recovered":56783,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:52:57.112 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:52:57.113 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:52:57.305 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:52:57.305 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:52:57.319 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":634,"stamp":"2026-02-22T20:52:57.122816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":46238,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56947,"num_bytes_recovered":270967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80137,"ondisk_log_size":80137,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1153356,"kb_used_data":485804,"kb_used_omap":49,"kb_used_meta":667470,"kb_avail":187590324,"statfs":{"total":193273528320,"available":192092491776,"internally_reserved":0,"allocated":497463296,"data_stored":805383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":50276,"internal_metadata":683490204},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10918,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5458,"num_bytes_recovered":5458,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000941"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":19209,"reported_epoch":41,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:52:52.288895+0000","last_change":"2026-02-22T20:52:51.210008+0000","last_active":"2026-02-22T20:52:52.288895+0000","last_peered":"2026-02-22T20:52:52.288895+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:52.288895+0000","last_undegraded":"2026-02-22T20:52:52.288895+0000","last_fullsized":"2026-02-22T20:52:52.288895+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14458,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2977,"num_bytes_recovered":2977,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41257,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.208446+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:52:51.208446+0000","last_peered":"2026-02-22T20:52:51.208446+0000","last_clean":"2026-02-22T20:52:51.208446+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:52:51.208446+0000","last_undegraded":"2026-02-22T20:52:51.208446+0000","last_fullsized":"2026-02-22T20:52:51.208446+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40501,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.202401+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:51.202401+0000","last_peered":"2026-02-22T20:52:51.202401+0000","last_clean":"2026-02-22T20:52:51.202401+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:51.202401+0000","last_undegraded":"2026-02-22T20:52:51.202401+0000","last_fullsized":"2026-02-22T20:52:51.202401+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41820,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.202356+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:51.202356+0000","last_peered":"2026-02-22T20:52:51.202356+0000","last_clean":"2026-02-22T20:52:51.202356+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:51.202356+0000","last_undegraded":"2026-02-22T20:52:51.202356+0000","last_fullsized":"2026-02-22T20:52:51.202356+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10122,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.203011+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:51.203011+0000","last_peered":"2026-02-22T20:52:51.203011+0000","last_clean":"2026-02-22T20:52:51.203011+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:51.203011+0000","last_undegraded":"2026-02-22T20:52:51.203011+0000","last_fullsized":"2026-02-22T20:52:51.203011+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40761,"reported_epoch":41,"state":"active+clean","last_fresh":"2026-02-22T20:52:51.202351+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:51.202351+0000","last_peered":"2026-02-22T20:52:51.202351+0000","last_clean":"2026-02-22T20:52:51.202351+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:51.202351+0000","last_undegraded":"2026-02-22T20:52:51.202351+0000","last_fullsized":"2026-02-22T20:52:51.202351+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":19593,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.210026+0000","last_change":"2026-02-22T20:52:51.209471+0000","last_active":"2026-02-22T20:52:51.210026+0000","last_peered":"2026-02-22T20:52:51.210026+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:51.210026+0000","last_undegraded":"2026-02-22T20:52:51.210026+0000","last_fullsized":"2026-02-22T20:52:51.210026+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14302,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3088,"num_bytes_recovered":3088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14818,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.201275+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:51.201275+0000","last_peered":"2026-02-22T20:52:51.201275+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:51.201275+0000","last_undegraded":"2026-02-22T20:52:51.201275+0000","last_fullsized":"2026-02-22T20:52:51.201275+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":46238,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56947,"num_bytes_recovered":270967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":951996416,"data_stored":981264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80137,"ondisk_log_size":80137,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051603,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182421,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705907,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576716,"kb_used_data":242940,"kb_used_omap":24,"kb_used_meta":333735,"kb_avail":93795124,"statfs":{"total":96636764160,"available":96046206976,"internally_reserved":0,"allocated":248770560,"data_stored":402701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":341745102},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576640,"kb_used_data":242864,"kb_used_omap":24,"kb_used_meta":333735,"kb_avail":93795200,"statfs":{"total":96636764160,"available":96046284800,"internally_reserved":0,"allocated":248692736,"data_stored":402682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":341745102},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":252473344,"data_stored":275600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252473344,"data_stored":275600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321314816,"data_stored":322967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":125734912,"data_stored":107097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:00.321 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:00.521 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:00.521 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:00.536 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":637,"stamp":"2026-02-22T20:53:00.219038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":31780,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64175,"num_bytes_recovered":278195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1268884,"kb_used_data":529140,"kb_used_omap":49,"kb_used_meta":739662,"kb_avail":187474796,"statfs":{"total":193273528320,"available":191974191104,"internally_reserved":0,"allocated":541839360,"data_stored":816217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":50276,"internal_metadata":757414812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-25376,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12686,"num_bytes_recovered":12686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.018515"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40895,"reported_epoch":42,"state":"peering","last_fresh":"2026-02-22T20:52:59.219378+0000","last_change":"2026-02-22T20:52:59.219378+0000","last_active":"2026-02-22T20:52:59.219378+0000","last_peered":"2026-02-22T20:52:58.957638+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:59.219378+0000","last_undegraded":"2026-02-22T20:52:59.219378+0000","last_fullsized":"2026-02-22T20:52:59.219378+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41259,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219107+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:52:59.219107+0000","last_peered":"2026-02-22T20:52:59.219107+0000","last_clean":"2026-02-22T20:52:59.219107+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:52:59.219107+0000","last_undegraded":"2026-02-22T20:52:59.219107+0000","last_fullsized":"2026-02-22T20:52:59.219107+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40503,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219184+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:59.219184+0000","last_peered":"2026-02-22T20:52:59.219184+0000","last_clean":"2026-02-22T20:52:59.219184+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:59.219184+0000","last_undegraded":"2026-02-22T20:52:59.219184+0000","last_fullsized":"2026-02-22T20:52:59.219184+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41822,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219098+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:59.219098+0000","last_peered":"2026-02-22T20:52:59.219098+0000","last_clean":"2026-02-22T20:52:59.219098+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:59.219098+0000","last_undegraded":"2026-02-22T20:52:59.219098+0000","last_fullsized":"2026-02-22T20:52:59.219098+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10124,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219006+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:59.219006+0000","last_peered":"2026-02-22T20:52:59.219006+0000","last_clean":"2026-02-22T20:52:59.219006+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:59.219006+0000","last_undegraded":"2026-02-22T20:52:59.219006+0000","last_fullsized":"2026-02-22T20:52:59.219006+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40763,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219085+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:59.219085+0000","last_peered":"2026-02-22T20:52:59.219085+0000","last_clean":"2026-02-22T20:52:59.219085+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:59.219085+0000","last_undegraded":"2026-02-22T20:52:59.219085+0000","last_fullsized":"2026-02-22T20:52:59.219085+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":19593,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.210026+0000","last_change":"2026-02-22T20:52:51.209471+0000","last_active":"2026-02-22T20:52:51.210026+0000","last_peered":"2026-02-22T20:52:51.210026+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:51.210026+0000","last_undegraded":"2026-02-22T20:52:51.210026+0000","last_fullsized":"2026-02-22T20:52:51.210026+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14302,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3088,"num_bytes_recovered":3088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14818,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.201275+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:51.201275+0000","last_peered":"2026-02-22T20:52:51.201275+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:51.201275+0000","last_undegraded":"2026-02-22T20:52:51.201275+0000","last_fullsized":"2026-02-22T20:52:51.201275+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":31780,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64175,"num_bytes_recovered":278195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":993255424,"data_stored":991337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051604,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182422,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634420,"kb_used_data":264548,"kb_used_omap":24,"kb_used_meta":369831,"kb_avail":93737420,"statfs":{"total":96636764160,"available":95987118080,"internally_reserved":0,"allocated":270897152,"data_stored":408103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":378707406},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634464,"kb_used_data":264592,"kb_used_omap":24,"kb_used_meta":369831,"kb_avail":93737376,"statfs":{"total":96636764160,"available":95987073024,"internally_reserved":0,"allocated":270942208,"data_stored":408114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":378707406},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":274640896,"data_stored":281012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274636800,"data_stored":281011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":318242816,"data_stored":322217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":125734912,"data_stored":107097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:00.537 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:00.727 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:00.727 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:00.740 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":637,"stamp":"2026-02-22T20:53:00.219038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":31780,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64175,"num_bytes_recovered":278195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1268884,"kb_used_data":529140,"kb_used_omap":49,"kb_used_meta":739662,"kb_avail":187474796,"statfs":{"total":193273528320,"available":191974191104,"internally_reserved":0,"allocated":541839360,"data_stored":816217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":50276,"internal_metadata":757414812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-25376,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12686,"num_bytes_recovered":12686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.018515"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40895,"reported_epoch":42,"state":"peering","last_fresh":"2026-02-22T20:52:59.219378+0000","last_change":"2026-02-22T20:52:59.219378+0000","last_active":"2026-02-22T20:52:59.219378+0000","last_peered":"2026-02-22T20:52:58.957638+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:59.219378+0000","last_undegraded":"2026-02-22T20:52:59.219378+0000","last_fullsized":"2026-02-22T20:52:59.219378+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41259,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219107+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:52:59.219107+0000","last_peered":"2026-02-22T20:52:59.219107+0000","last_clean":"2026-02-22T20:52:59.219107+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:52:59.219107+0000","last_undegraded":"2026-02-22T20:52:59.219107+0000","last_fullsized":"2026-02-22T20:52:59.219107+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40503,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219184+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:59.219184+0000","last_peered":"2026-02-22T20:52:59.219184+0000","last_clean":"2026-02-22T20:52:59.219184+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:59.219184+0000","last_undegraded":"2026-02-22T20:52:59.219184+0000","last_fullsized":"2026-02-22T20:52:59.219184+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41822,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219098+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:59.219098+0000","last_peered":"2026-02-22T20:52:59.219098+0000","last_clean":"2026-02-22T20:52:59.219098+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:59.219098+0000","last_undegraded":"2026-02-22T20:52:59.219098+0000","last_fullsized":"2026-02-22T20:52:59.219098+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10124,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219006+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:59.219006+0000","last_peered":"2026-02-22T20:52:59.219006+0000","last_clean":"2026-02-22T20:52:59.219006+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:59.219006+0000","last_undegraded":"2026-02-22T20:52:59.219006+0000","last_fullsized":"2026-02-22T20:52:59.219006+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40763,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219085+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:59.219085+0000","last_peered":"2026-02-22T20:52:59.219085+0000","last_clean":"2026-02-22T20:52:59.219085+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:59.219085+0000","last_undegraded":"2026-02-22T20:52:59.219085+0000","last_fullsized":"2026-02-22T20:52:59.219085+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":19593,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.210026+0000","last_change":"2026-02-22T20:52:51.209471+0000","last_active":"2026-02-22T20:52:51.210026+0000","last_peered":"2026-02-22T20:52:51.210026+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:51.210026+0000","last_undegraded":"2026-02-22T20:52:51.210026+0000","last_fullsized":"2026-02-22T20:52:51.210026+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14302,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3088,"num_bytes_recovered":3088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14818,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.201275+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:51.201275+0000","last_peered":"2026-02-22T20:52:51.201275+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:51.201275+0000","last_undegraded":"2026-02-22T20:52:51.201275+0000","last_fullsized":"2026-02-22T20:52:51.201275+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":31780,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64175,"num_bytes_recovered":278195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":993255424,"data_stored":991337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051604,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182422,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634420,"kb_used_data":264548,"kb_used_omap":24,"kb_used_meta":369831,"kb_avail":93737420,"statfs":{"total":96636764160,"available":95987118080,"internally_reserved":0,"allocated":270897152,"data_stored":408103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":378707406},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634464,"kb_used_data":264592,"kb_used_omap":24,"kb_used_meta":369831,"kb_avail":93737376,"statfs":{"total":96636764160,"available":95987073024,"internally_reserved":0,"allocated":270942208,"data_stored":408114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":378707406},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":274640896,"data_stored":281012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274636800,"data_stored":281011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":318242816,"data_stored":322217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":125734912,"data_stored":107097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:00.741 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:53:00.741 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 40895, 'reported_epoch': 42, 'state': 'peering', 'last_fresh': '2026-02-22T20:52:59.219378+0000', 'last_change': '2026-02-22T20:52:59.219378+0000', 'last_active': '2026-02-22T20:52:59.219378+0000', 'last_peered': '2026-02-22T20:52:58.957638+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:59.219378+0000', 'last_undegraded': '2026-02-22T20:52:59.219378+0000', 'last_fullsized': '2026-02-22T20:52:59.219378+0000', 'mapping_epoch': 42, 'log_start': "17'205", 'ondisk_log_start': "17'205", 'created': 10, 'last_epoch_clean': 41, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10000, 'log_dups_size': 0, 'ondisk_log_size': 10000, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10205, 'num_bytes_recovered': 10205, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2026-02-22T20:53:00.741 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:53:00.741 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 19593, 'reported_epoch': 41, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:51.210026+0000', 'last_change': '2026-02-22T20:52:51.209471+0000', 'last_active': '2026-02-22T20:52:51.210026+0000', 'last_peered': '2026-02-22T20:52:51.210026+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:52:51.210026+0000', 'last_undegraded': '2026-02-22T20:52:51.210026+0000', 'last_fullsized': '2026-02-22T20:52:51.210026+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 14302, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3088, 'num_bytes_recovered': 3088, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:53:00.741 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:53:00.741 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 14818, 'reported_epoch': 41, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:52:51.201275+0000', 'last_change': '2026-02-22T20:52:42.196267+0000', 'last_active': '2026-02-22T20:52:51.201275+0000', 'last_peered': '2026-02-22T20:52:51.201275+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:52:51.201275+0000', 'last_undegraded': '2026-02-22T20:52:51.201275+0000', 'last_fullsized': '2026-02-22T20:52:51.201275+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 17478, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1501, 'num_bytes_recovered': 1501, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:53:00.741 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:53:00.973 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:00.973 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1197,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":43,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":5},{"state_name":"active+remapped+backfill_wait","count":2},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":1181036544,"bytes_avail":192092491776,"bytes_total":193273528320,"misplaced_objects":35426,"misplaced_total":163508,"misplaced_ratio":0.21666218166695209,"recovering_objects_per_sec":1217,"recovering_bytes_per_sec":1217,"recovering_keys_per_sec":0,"num_objects_recovered":10864,"num_bytes_recovered":10864,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:53:00.988 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:53:00.989 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:01.177 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:01.178 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:01.191 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":638,"stamp":"2026-02-22T20:53:01.123296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":31780,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64175,"num_bytes_recovered":278195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1268884,"kb_used_data":529140,"kb_used_omap":49,"kb_used_meta":739662,"kb_avail":187474796,"statfs":{"total":193273528320,"available":191974191104,"internally_reserved":0,"allocated":541839360,"data_stored":816217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":50276,"internal_metadata":757414812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-20632,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10314,"num_bytes_recovered":10314,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000964"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40895,"reported_epoch":42,"state":"peering","last_fresh":"2026-02-22T20:52:59.219378+0000","last_change":"2026-02-22T20:52:59.219378+0000","last_active":"2026-02-22T20:52:59.219378+0000","last_peered":"2026-02-22T20:52:58.957638+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:59.219378+0000","last_undegraded":"2026-02-22T20:52:59.219378+0000","last_fullsized":"2026-02-22T20:52:59.219378+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41259,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219107+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:52:59.219107+0000","last_peered":"2026-02-22T20:52:59.219107+0000","last_clean":"2026-02-22T20:52:59.219107+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:52:59.219107+0000","last_undegraded":"2026-02-22T20:52:59.219107+0000","last_fullsized":"2026-02-22T20:52:59.219107+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40503,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219184+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:59.219184+0000","last_peered":"2026-02-22T20:52:59.219184+0000","last_clean":"2026-02-22T20:52:59.219184+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:59.219184+0000","last_undegraded":"2026-02-22T20:52:59.219184+0000","last_fullsized":"2026-02-22T20:52:59.219184+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41822,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219098+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:59.219098+0000","last_peered":"2026-02-22T20:52:59.219098+0000","last_clean":"2026-02-22T20:52:59.219098+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:59.219098+0000","last_undegraded":"2026-02-22T20:52:59.219098+0000","last_fullsized":"2026-02-22T20:52:59.219098+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10124,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219006+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:59.219006+0000","last_peered":"2026-02-22T20:52:59.219006+0000","last_clean":"2026-02-22T20:52:59.219006+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:59.219006+0000","last_undegraded":"2026-02-22T20:52:59.219006+0000","last_fullsized":"2026-02-22T20:52:59.219006+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40763,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219085+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:59.219085+0000","last_peered":"2026-02-22T20:52:59.219085+0000","last_clean":"2026-02-22T20:52:59.219085+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:59.219085+0000","last_undegraded":"2026-02-22T20:52:59.219085+0000","last_fullsized":"2026-02-22T20:52:59.219085+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":19593,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.210026+0000","last_change":"2026-02-22T20:52:51.209471+0000","last_active":"2026-02-22T20:52:51.210026+0000","last_peered":"2026-02-22T20:52:51.210026+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:52:51.210026+0000","last_undegraded":"2026-02-22T20:52:51.210026+0000","last_fullsized":"2026-02-22T20:52:51.210026+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14302,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3088,"num_bytes_recovered":3088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":14818,"reported_epoch":41,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:52:51.201275+0000","last_change":"2026-02-22T20:52:42.196267+0000","last_active":"2026-02-22T20:52:51.201275+0000","last_peered":"2026-02-22T20:52:51.201275+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:52:51.201275+0000","last_undegraded":"2026-02-22T20:52:51.201275+0000","last_fullsized":"2026-02-22T20:52:51.201275+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17478,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":1501,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":31780,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64175,"num_bytes_recovered":278195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":993255424,"data_stored":991337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051604,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182422,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634420,"kb_used_data":264548,"kb_used_omap":24,"kb_used_meta":369831,"kb_avail":93737420,"statfs":{"total":96636764160,"available":95987118080,"internally_reserved":0,"allocated":270897152,"data_stored":408103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":378707406},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634464,"kb_used_data":264592,"kb_used_omap":24,"kb_used_meta":369831,"kb_avail":93737376,"statfs":{"total":96636764160,"available":95987073024,"internally_reserved":0,"allocated":270942208,"data_stored":408114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":378707406},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":274640896,"data_stored":281012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274636800,"data_stored":281011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":318242816,"data_stored":322217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":125734912,"data_stored":107097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:04.193 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:04.389 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:04.389 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:04.403 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":639,"stamp":"2026-02-22T20:53:03.123513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":24640,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67745,"num_bytes_recovered":297048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1268884,"kb_used_data":529140,"kb_used_omap":49,"kb_used_meta":739662,"kb_avail":187474796,"statfs":{"total":193273528320,"available":191974191104,"internally_reserved":0,"allocated":541839360,"data_stored":816217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":50276,"internal_metadata":757414812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21598,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10798,"num_bytes_recovered":26081,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000966"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40895,"reported_epoch":42,"state":"peering","last_fresh":"2026-02-22T20:52:59.219378+0000","last_change":"2026-02-22T20:52:59.219378+0000","last_active":"2026-02-22T20:52:59.219378+0000","last_peered":"2026-02-22T20:52:58.957638+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:59.219378+0000","last_undegraded":"2026-02-22T20:52:59.219378+0000","last_fullsized":"2026-02-22T20:52:59.219378+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41259,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219107+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:52:59.219107+0000","last_peered":"2026-02-22T20:52:59.219107+0000","last_clean":"2026-02-22T20:52:59.219107+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:52:59.219107+0000","last_undegraded":"2026-02-22T20:52:59.219107+0000","last_fullsized":"2026-02-22T20:52:59.219107+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40503,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219184+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:59.219184+0000","last_peered":"2026-02-22T20:52:59.219184+0000","last_clean":"2026-02-22T20:52:59.219184+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:59.219184+0000","last_undegraded":"2026-02-22T20:52:59.219184+0000","last_fullsized":"2026-02-22T20:52:59.219184+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41822,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219098+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:59.219098+0000","last_peered":"2026-02-22T20:52:59.219098+0000","last_clean":"2026-02-22T20:52:59.219098+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:59.219098+0000","last_undegraded":"2026-02-22T20:52:59.219098+0000","last_fullsized":"2026-02-22T20:52:59.219098+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10124,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219006+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:59.219006+0000","last_peered":"2026-02-22T20:52:59.219006+0000","last_clean":"2026-02-22T20:52:59.219006+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:59.219006+0000","last_undegraded":"2026-02-22T20:52:59.219006+0000","last_fullsized":"2026-02-22T20:52:59.219006+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40763,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219085+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:59.219085+0000","last_peered":"2026-02-22T20:52:59.219085+0000","last_clean":"2026-02-22T20:52:59.219085+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:59.219085+0000","last_undegraded":"2026-02-22T20:52:59.219085+0000","last_fullsized":"2026-02-22T20:52:59.219085+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":26193,"reported_epoch":43,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:53:02.289907+0000","last_change":"2026-02-22T20:53:00.228605+0000","last_active":"2026-02-22T20:53:02.289907+0000","last_peered":"2026-02-22T20:53:02.289907+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:53:02.289907+0000","last_undegraded":"2026-02-22T20:53:02.289907+0000","last_fullsized":"2026-02-22T20:53:02.289907+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9906,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5286,"num_bytes_recovered":20569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":18942,"reported_epoch":43,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:53:00.228463+0000","last_change":"2026-02-22T20:53:00.227995+0000","last_active":"2026-02-22T20:53:00.228463+0000","last_peered":"2026-02-22T20:53:00.228463+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:00.228463+0000","last_undegraded":"2026-02-22T20:53:00.228463+0000","last_fullsized":"2026-02-22T20:53:00.228463+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14734,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2873,"num_bytes_recovered":2873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":24640,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67745,"num_bytes_recovered":297048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":989822976,"data_stored":990499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051604,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182423,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634420,"kb_used_data":264548,"kb_used_omap":24,"kb_used_meta":369831,"kb_avail":93737420,"statfs":{"total":96636764160,"available":95987118080,"internally_reserved":0,"allocated":270897152,"data_stored":408103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":378707406},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634464,"kb_used_data":264592,"kb_used_omap":24,"kb_used_meta":369831,"kb_avail":93737376,"statfs":{"total":96636764160,"available":95987073024,"internally_reserved":0,"allocated":270942208,"data_stored":408114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":378707406},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":274640896,"data_stored":281012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274636800,"data_stored":281011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":314810368,"data_stored":321379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":125734912,"data_stored":107097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:04.405 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:04.593 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:04.594 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:04.608 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":639,"stamp":"2026-02-22T20:53:03.123513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":24640,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67745,"num_bytes_recovered":297048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1268884,"kb_used_data":529140,"kb_used_omap":49,"kb_used_meta":739662,"kb_avail":187474796,"statfs":{"total":193273528320,"available":191974191104,"internally_reserved":0,"allocated":541839360,"data_stored":816217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":50276,"internal_metadata":757414812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21598,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10798,"num_bytes_recovered":26081,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000966"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40895,"reported_epoch":42,"state":"peering","last_fresh":"2026-02-22T20:52:59.219378+0000","last_change":"2026-02-22T20:52:59.219378+0000","last_active":"2026-02-22T20:52:59.219378+0000","last_peered":"2026-02-22T20:52:58.957638+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:59.219378+0000","last_undegraded":"2026-02-22T20:52:59.219378+0000","last_fullsized":"2026-02-22T20:52:59.219378+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41259,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219107+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:52:59.219107+0000","last_peered":"2026-02-22T20:52:59.219107+0000","last_clean":"2026-02-22T20:52:59.219107+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:52:59.219107+0000","last_undegraded":"2026-02-22T20:52:59.219107+0000","last_fullsized":"2026-02-22T20:52:59.219107+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40503,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219184+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:59.219184+0000","last_peered":"2026-02-22T20:52:59.219184+0000","last_clean":"2026-02-22T20:52:59.219184+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:59.219184+0000","last_undegraded":"2026-02-22T20:52:59.219184+0000","last_fullsized":"2026-02-22T20:52:59.219184+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41822,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219098+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:59.219098+0000","last_peered":"2026-02-22T20:52:59.219098+0000","last_clean":"2026-02-22T20:52:59.219098+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:59.219098+0000","last_undegraded":"2026-02-22T20:52:59.219098+0000","last_fullsized":"2026-02-22T20:52:59.219098+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10124,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219006+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:59.219006+0000","last_peered":"2026-02-22T20:52:59.219006+0000","last_clean":"2026-02-22T20:52:59.219006+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:59.219006+0000","last_undegraded":"2026-02-22T20:52:59.219006+0000","last_fullsized":"2026-02-22T20:52:59.219006+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40763,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219085+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:59.219085+0000","last_peered":"2026-02-22T20:52:59.219085+0000","last_clean":"2026-02-22T20:52:59.219085+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:59.219085+0000","last_undegraded":"2026-02-22T20:52:59.219085+0000","last_fullsized":"2026-02-22T20:52:59.219085+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":26193,"reported_epoch":43,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:53:02.289907+0000","last_change":"2026-02-22T20:53:00.228605+0000","last_active":"2026-02-22T20:53:02.289907+0000","last_peered":"2026-02-22T20:53:02.289907+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:53:02.289907+0000","last_undegraded":"2026-02-22T20:53:02.289907+0000","last_fullsized":"2026-02-22T20:53:02.289907+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9906,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5286,"num_bytes_recovered":20569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":18942,"reported_epoch":43,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:53:00.228463+0000","last_change":"2026-02-22T20:53:00.227995+0000","last_active":"2026-02-22T20:53:00.228463+0000","last_peered":"2026-02-22T20:53:00.228463+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:00.228463+0000","last_undegraded":"2026-02-22T20:53:00.228463+0000","last_fullsized":"2026-02-22T20:53:00.228463+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14734,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2873,"num_bytes_recovered":2873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":24640,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67745,"num_bytes_recovered":297048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":989822976,"data_stored":990499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051604,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182423,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634420,"kb_used_data":264548,"kb_used_omap":24,"kb_used_meta":369831,"kb_avail":93737420,"statfs":{"total":96636764160,"available":95987118080,"internally_reserved":0,"allocated":270897152,"data_stored":408103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":378707406},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634464,"kb_used_data":264592,"kb_used_omap":24,"kb_used_meta":369831,"kb_avail":93737376,"statfs":{"total":96636764160,"available":95987073024,"internally_reserved":0,"allocated":270942208,"data_stored":408114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":378707406},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":274640896,"data_stored":281012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274636800,"data_stored":281011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":314810368,"data_stored":321379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":125734912,"data_stored":107097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:04.609 INFO:tasks.osd_recovery.ceph_manager:PG 1.7 is not active+clean 2026-02-22T20:53:04.609 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.7', 'version': "17'10205", 'reported_seq': 40895, 'reported_epoch': 42, 'state': 'peering', 'last_fresh': '2026-02-22T20:52:59.219378+0000', 'last_change': '2026-02-22T20:52:59.219378+0000', 'last_active': '2026-02-22T20:52:59.219378+0000', 'last_peered': '2026-02-22T20:52:58.957638+0000', 'last_clean': '2026-02-22T20:50:28.895219+0000', 'last_became_active': '2026-02-22T20:51:26.182245+0000', 'last_became_peered': '2026-02-22T20:51:26.182245+0000', 'last_unstale': '2026-02-22T20:52:59.219378+0000', 'last_undegraded': '2026-02-22T20:52:59.219378+0000', 'last_fullsized': '2026-02-22T20:52:59.219378+0000', 'mapping_epoch': 42, 'log_start': "17'205", 'ondisk_log_start': "17'205", 'created': 10, 'last_epoch_clean': 41, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10000, 'log_dups_size': 0, 'ondisk_log_size': 10000, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10205, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10205, 'num_write_kb': 10205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10205, 'num_bytes_recovered': 10205, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2026-02-22T20:53:04.610 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:53:04.610 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 26193, 'reported_epoch': 43, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:53:02.289907+0000', 'last_change': '2026-02-22T20:53:00.228605+0000', 'last_active': '2026-02-22T20:53:02.289907+0000', 'last_peered': '2026-02-22T20:53:02.289907+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:53:02.289907+0000', 'last_undegraded': '2026-02-22T20:53:02.289907+0000', 'last_fullsized': '2026-02-22T20:53:02.289907+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 9906, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5286, 'num_bytes_recovered': 20569, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:53:04.610 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:53:04.610 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 18942, 'reported_epoch': 43, 'state': 'active+remapped+backfill_wait', 'last_fresh': '2026-02-22T20:53:00.228463+0000', 'last_change': '2026-02-22T20:53:00.227995+0000', 'last_active': '2026-02-22T20:53:00.228463+0000', 'last_peered': '2026-02-22T20:53:00.228463+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:53:00.228463+0000', 'last_undegraded': '2026-02-22T20:53:00.228463+0000', 'last_fullsized': '2026-02-22T20:53:00.228463+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 14734, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2873, 'num_bytes_recovered': 2873, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:53:04.611 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:53:04.844 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:04.844 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1201,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":43,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":5},{"state_name":"active+remapped+backfill_wait","count":1},{"state_name":"peering","count":1},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":1299337216,"bytes_avail":191974191104,"bytes_total":193273528320,"inactive_pgs_ratio":0.125,"misplaced_objects":24640,"misplaced_total":163508,"misplaced_ratio":0.15069599041025517,"recovering_objects_per_sec":1349,"recovering_bytes_per_sec":3259,"recovering_keys_per_sec":0,"num_objects_recovered":10798,"num_bytes_recovered":26081,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:53:04.860 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:53:04.860 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:05.050 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:05.051 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:05.064 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":639,"stamp":"2026-02-22T20:53:03.123513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":24640,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67745,"num_bytes_recovered":297048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1268884,"kb_used_data":529140,"kb_used_omap":49,"kb_used_meta":739662,"kb_avail":187474796,"statfs":{"total":193273528320,"available":191974191104,"internally_reserved":0,"allocated":541839360,"data_stored":816217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":50276,"internal_metadata":757414812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-21598,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10798,"num_bytes_recovered":26081,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000966"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40895,"reported_epoch":42,"state":"peering","last_fresh":"2026-02-22T20:52:59.219378+0000","last_change":"2026-02-22T20:52:59.219378+0000","last_active":"2026-02-22T20:52:59.219378+0000","last_peered":"2026-02-22T20:52:58.957638+0000","last_clean":"2026-02-22T20:50:28.895219+0000","last_became_active":"2026-02-22T20:51:26.182245+0000","last_became_peered":"2026-02-22T20:51:26.182245+0000","last_unstale":"2026-02-22T20:52:59.219378+0000","last_undegraded":"2026-02-22T20:52:59.219378+0000","last_fullsized":"2026-02-22T20:52:59.219378+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41259,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219107+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:52:59.219107+0000","last_peered":"2026-02-22T20:52:59.219107+0000","last_clean":"2026-02-22T20:52:59.219107+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:52:59.219107+0000","last_undegraded":"2026-02-22T20:52:59.219107+0000","last_fullsized":"2026-02-22T20:52:59.219107+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40503,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219184+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:52:59.219184+0000","last_peered":"2026-02-22T20:52:59.219184+0000","last_clean":"2026-02-22T20:52:59.219184+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:52:59.219184+0000","last_undegraded":"2026-02-22T20:52:59.219184+0000","last_fullsized":"2026-02-22T20:52:59.219184+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41822,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219098+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:52:59.219098+0000","last_peered":"2026-02-22T20:52:59.219098+0000","last_clean":"2026-02-22T20:52:59.219098+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:52:59.219098+0000","last_undegraded":"2026-02-22T20:52:59.219098+0000","last_fullsized":"2026-02-22T20:52:59.219098+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10124,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219006+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:52:59.219006+0000","last_peered":"2026-02-22T20:52:59.219006+0000","last_clean":"2026-02-22T20:52:59.219006+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:52:59.219006+0000","last_undegraded":"2026-02-22T20:52:59.219006+0000","last_fullsized":"2026-02-22T20:52:59.219006+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40763,"reported_epoch":42,"state":"active+clean","last_fresh":"2026-02-22T20:52:59.219085+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:52:59.219085+0000","last_peered":"2026-02-22T20:52:59.219085+0000","last_clean":"2026-02-22T20:52:59.219085+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:52:59.219085+0000","last_undegraded":"2026-02-22T20:52:59.219085+0000","last_fullsized":"2026-02-22T20:52:59.219085+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":26193,"reported_epoch":43,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:53:02.289907+0000","last_change":"2026-02-22T20:53:00.228605+0000","last_active":"2026-02-22T20:53:02.289907+0000","last_peered":"2026-02-22T20:53:02.289907+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:53:02.289907+0000","last_undegraded":"2026-02-22T20:53:02.289907+0000","last_fullsized":"2026-02-22T20:53:02.289907+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9906,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5286,"num_bytes_recovered":20569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":18942,"reported_epoch":43,"state":"active+remapped+backfill_wait","last_fresh":"2026-02-22T20:53:00.228463+0000","last_change":"2026-02-22T20:53:00.227995+0000","last_active":"2026-02-22T20:53:00.228463+0000","last_peered":"2026-02-22T20:53:00.228463+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:00.228463+0000","last_undegraded":"2026-02-22T20:53:00.228463+0000","last_fullsized":"2026-02-22T20:53:00.228463+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14734,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2873,"num_bytes_recovered":2873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":24640,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67745,"num_bytes_recovered":297048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":989822976,"data_stored":990499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051604,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182423,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705908,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634420,"kb_used_data":264548,"kb_used_omap":24,"kb_used_meta":369831,"kb_avail":93737420,"statfs":{"total":96636764160,"available":95987118080,"internally_reserved":0,"allocated":270897152,"data_stored":408103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":378707406},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634464,"kb_used_data":264592,"kb_used_omap":24,"kb_used_meta":369831,"kb_avail":93737376,"statfs":{"total":96636764160,"available":95987073024,"internally_reserved":0,"allocated":270942208,"data_stored":408114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":378707406},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":274640896,"data_stored":281012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274636800,"data_stored":281011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":314810368,"data_stored":321379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":125734912,"data_stored":107097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:08.065 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:08.259 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:08.259 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:08.272 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":643,"stamp":"2026-02-22T20:53:08.234993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":23616,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68257,"num_bytes_recovered":297560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1333892,"kb_used_data":576100,"kb_used_omap":51,"kb_used_meta":757708,"kb_avail":187409788,"statfs":{"total":193273528320,"available":191907622912,"internally_reserved":0,"allocated":589926400,"data_stored":871543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52880,"internal_metadata":775893360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-8164,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4082,"num_bytes_recovered":19365,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.015954"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40904,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.227657+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:00.227657+0000","last_peered":"2026-02-22T20:53:00.227657+0000","last_clean":"2026-02-22T20:53:00.227657+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:00.227657+0000","last_undegraded":"2026-02-22T20:53:00.227657+0000","last_fullsized":"2026-02-22T20:53:00.227657+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41261,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.222083+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:00.222083+0000","last_peered":"2026-02-22T20:53:00.222083+0000","last_clean":"2026-02-22T20:53:00.222083+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:00.222083+0000","last_undegraded":"2026-02-22T20:53:00.222083+0000","last_fullsized":"2026-02-22T20:53:00.222083+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40505,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.222128+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:00.222128+0000","last_peered":"2026-02-22T20:53:00.222128+0000","last_clean":"2026-02-22T20:53:00.222128+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:00.222128+0000","last_undegraded":"2026-02-22T20:53:00.222128+0000","last_fullsized":"2026-02-22T20:53:00.222128+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41824,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.222046+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:00.222046+0000","last_peered":"2026-02-22T20:53:00.222046+0000","last_clean":"2026-02-22T20:53:00.222046+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:00.222046+0000","last_undegraded":"2026-02-22T20:53:00.222046+0000","last_fullsized":"2026-02-22T20:53:00.222046+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10126,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.220740+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:00.220740+0000","last_peered":"2026-02-22T20:53:00.220740+0000","last_clean":"2026-02-22T20:53:00.220740+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:00.220740+0000","last_undegraded":"2026-02-22T20:53:00.220740+0000","last_fullsized":"2026-02-22T20:53:00.220740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40765,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.222002+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:00.222002+0000","last_peered":"2026-02-22T20:53:00.222002+0000","last_clean":"2026-02-22T20:53:00.222002+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:00.222002+0000","last_undegraded":"2026-02-22T20:53:00.222002+0000","last_fullsized":"2026-02-22T20:53:00.222002+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":26193,"reported_epoch":43,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:53:02.289907+0000","last_change":"2026-02-22T20:53:00.228605+0000","last_active":"2026-02-22T20:53:02.289907+0000","last_peered":"2026-02-22T20:53:02.289907+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:53:02.289907+0000","last_undegraded":"2026-02-22T20:53:02.289907+0000","last_fullsized":"2026-02-22T20:53:02.289907+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9906,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5286,"num_bytes_recovered":20569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":20482,"reported_epoch":44,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:53:07.289948+0000","last_change":"2026-02-22T20:53:06.816383+0000","last_active":"2026-02-22T20:53:07.289948+0000","last_peered":"2026-02-22T20:53:07.289948+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:07.289948+0000","last_undegraded":"2026-02-22T20:53:07.289948+0000","last_fullsized":"2026-02-22T20:53:07.289948+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13710,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3385,"num_bytes_recovered":3385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":23616,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68257,"num_bytes_recovered":297560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":988635136,"data_stored":1005489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051606,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182424,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677816,"kb_used_data":289896,"kb_used_omap":25,"kb_used_meta":387878,"kb_avail":93694024,"statfs":{"total":96636764160,"available":95942680576,"internally_reserved":0,"allocated":296853504,"data_stored":436233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":397187256},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705910,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656076,"kb_used_data":286204,"kb_used_omap":25,"kb_used_meta":369830,"kb_avail":93715764,"statfs":{"total":96636764160,"available":95964942336,"internally_reserved":0,"allocated":293072896,"data_stored":435310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":378706104},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":296796160,"data_stored":301701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296792064,"data_stored":301700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311111680,"data_stored":305196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":83935232,"data_stored":96892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:08.273 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:08.464 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:08.464 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:08.478 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":643,"stamp":"2026-02-22T20:53:08.234993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":23616,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68257,"num_bytes_recovered":297560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1333892,"kb_used_data":576100,"kb_used_omap":51,"kb_used_meta":757708,"kb_avail":187409788,"statfs":{"total":193273528320,"available":191907622912,"internally_reserved":0,"allocated":589926400,"data_stored":871543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52880,"internal_metadata":775893360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-8164,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4082,"num_bytes_recovered":19365,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.015954"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40904,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.227657+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:00.227657+0000","last_peered":"2026-02-22T20:53:00.227657+0000","last_clean":"2026-02-22T20:53:00.227657+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:00.227657+0000","last_undegraded":"2026-02-22T20:53:00.227657+0000","last_fullsized":"2026-02-22T20:53:00.227657+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41261,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.222083+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:00.222083+0000","last_peered":"2026-02-22T20:53:00.222083+0000","last_clean":"2026-02-22T20:53:00.222083+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:00.222083+0000","last_undegraded":"2026-02-22T20:53:00.222083+0000","last_fullsized":"2026-02-22T20:53:00.222083+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40505,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.222128+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:00.222128+0000","last_peered":"2026-02-22T20:53:00.222128+0000","last_clean":"2026-02-22T20:53:00.222128+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:00.222128+0000","last_undegraded":"2026-02-22T20:53:00.222128+0000","last_fullsized":"2026-02-22T20:53:00.222128+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41824,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.222046+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:00.222046+0000","last_peered":"2026-02-22T20:53:00.222046+0000","last_clean":"2026-02-22T20:53:00.222046+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:00.222046+0000","last_undegraded":"2026-02-22T20:53:00.222046+0000","last_fullsized":"2026-02-22T20:53:00.222046+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10126,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.220740+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:00.220740+0000","last_peered":"2026-02-22T20:53:00.220740+0000","last_clean":"2026-02-22T20:53:00.220740+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:00.220740+0000","last_undegraded":"2026-02-22T20:53:00.220740+0000","last_fullsized":"2026-02-22T20:53:00.220740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40765,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.222002+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:00.222002+0000","last_peered":"2026-02-22T20:53:00.222002+0000","last_clean":"2026-02-22T20:53:00.222002+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:00.222002+0000","last_undegraded":"2026-02-22T20:53:00.222002+0000","last_fullsized":"2026-02-22T20:53:00.222002+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":26193,"reported_epoch":43,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:53:02.289907+0000","last_change":"2026-02-22T20:53:00.228605+0000","last_active":"2026-02-22T20:53:02.289907+0000","last_peered":"2026-02-22T20:53:02.289907+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:53:02.289907+0000","last_undegraded":"2026-02-22T20:53:02.289907+0000","last_fullsized":"2026-02-22T20:53:02.289907+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9906,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5286,"num_bytes_recovered":20569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":20482,"reported_epoch":44,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:53:07.289948+0000","last_change":"2026-02-22T20:53:06.816383+0000","last_active":"2026-02-22T20:53:07.289948+0000","last_peered":"2026-02-22T20:53:07.289948+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:07.289948+0000","last_undegraded":"2026-02-22T20:53:07.289948+0000","last_fullsized":"2026-02-22T20:53:07.289948+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13710,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3385,"num_bytes_recovered":3385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":23616,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68257,"num_bytes_recovered":297560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":988635136,"data_stored":1005489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051606,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182424,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677816,"kb_used_data":289896,"kb_used_omap":25,"kb_used_meta":387878,"kb_avail":93694024,"statfs":{"total":96636764160,"available":95942680576,"internally_reserved":0,"allocated":296853504,"data_stored":436233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":397187256},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705910,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656076,"kb_used_data":286204,"kb_used_omap":25,"kb_used_meta":369830,"kb_avail":93715764,"statfs":{"total":96636764160,"available":95964942336,"internally_reserved":0,"allocated":293072896,"data_stored":435310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":378706104},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":296796160,"data_stored":301701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296792064,"data_stored":301700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311111680,"data_stored":305196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":83935232,"data_stored":96892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:08.480 INFO:tasks.osd_recovery.ceph_manager:PG 1.1 is not active+clean 2026-02-22T20:53:08.480 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.1', 'version': "17'10238", 'reported_seq': 26193, 'reported_epoch': 43, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:53:02.289907+0000', 'last_change': '2026-02-22T20:53:00.228605+0000', 'last_active': '2026-02-22T20:53:02.289907+0000', 'last_peered': '2026-02-22T20:53:02.289907+0000', 'last_clean': '2026-02-22T20:50:28.894558+0000', 'last_became_active': '2026-02-22T20:51:26.182382+0000', 'last_became_peered': '2026-02-22T20:51:26.182382+0000', 'last_unstale': '2026-02-22T20:53:02.289907+0000', 'last_undegraded': '2026-02-22T20:53:02.289907+0000', 'last_fullsized': '2026-02-22T20:53:02.289907+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000658762, 'stat_sum': {'num_bytes': 25521, 'num_objects': 10238, 'num_object_clones': 0, 'num_object_copies': 20476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 9906, 'num_objects_unfound': 0, 'num_objects_dirty': 10238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10238, 'num_write_kb': 10252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5286, 'num_bytes_recovered': 20569, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:53:08.480 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:53:08.480 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 20482, 'reported_epoch': 44, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:53:07.289948+0000', 'last_change': '2026-02-22T20:53:06.816383+0000', 'last_active': '2026-02-22T20:53:07.289948+0000', 'last_peered': '2026-02-22T20:53:07.289948+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:53:07.289948+0000', 'last_undegraded': '2026-02-22T20:53:07.289948+0000', 'last_fullsized': '2026-02-22T20:53:07.289948+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 13710, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3385, 'num_bytes_recovered': 3385, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:53:08.481 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:53:08.717 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:08.717 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1205,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":45,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":6},{"state_name":"active+remapped+backfill_wait","count":1},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":1365905408,"bytes_avail":191907622912,"bytes_total":193273528320,"misplaced_objects":24640,"misplaced_total":163508,"misplaced_ratio":0.15069599041025517,"recovering_objects_per_sec":673,"recovering_bytes_per_sec":2584,"recovering_keys_per_sec":0,"num_objects_recovered":5392,"num_bytes_recovered":20675,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:53:08.732 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:53:08.732 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:08.922 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:08.922 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:08.939 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":643,"stamp":"2026-02-22T20:53:08.234993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":23616,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68257,"num_bytes_recovered":297560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1333892,"kb_used_data":576100,"kb_used_omap":51,"kb_used_meta":757708,"kb_avail":187409788,"statfs":{"total":193273528320,"available":191907622912,"internally_reserved":0,"allocated":589926400,"data_stored":871543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52880,"internal_metadata":775893360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-8164,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4082,"num_bytes_recovered":19365,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.015954"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40904,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.227657+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:00.227657+0000","last_peered":"2026-02-22T20:53:00.227657+0000","last_clean":"2026-02-22T20:53:00.227657+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:00.227657+0000","last_undegraded":"2026-02-22T20:53:00.227657+0000","last_fullsized":"2026-02-22T20:53:00.227657+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41261,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.222083+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:00.222083+0000","last_peered":"2026-02-22T20:53:00.222083+0000","last_clean":"2026-02-22T20:53:00.222083+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:00.222083+0000","last_undegraded":"2026-02-22T20:53:00.222083+0000","last_fullsized":"2026-02-22T20:53:00.222083+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40505,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.222128+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:00.222128+0000","last_peered":"2026-02-22T20:53:00.222128+0000","last_clean":"2026-02-22T20:53:00.222128+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:00.222128+0000","last_undegraded":"2026-02-22T20:53:00.222128+0000","last_fullsized":"2026-02-22T20:53:00.222128+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41824,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.222046+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:00.222046+0000","last_peered":"2026-02-22T20:53:00.222046+0000","last_clean":"2026-02-22T20:53:00.222046+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:00.222046+0000","last_undegraded":"2026-02-22T20:53:00.222046+0000","last_fullsized":"2026-02-22T20:53:00.222046+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10126,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.220740+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:00.220740+0000","last_peered":"2026-02-22T20:53:00.220740+0000","last_clean":"2026-02-22T20:53:00.220740+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:00.220740+0000","last_undegraded":"2026-02-22T20:53:00.220740+0000","last_fullsized":"2026-02-22T20:53:00.220740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40765,"reported_epoch":43,"state":"active+clean","last_fresh":"2026-02-22T20:53:00.222002+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:00.222002+0000","last_peered":"2026-02-22T20:53:00.222002+0000","last_clean":"2026-02-22T20:53:00.222002+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:00.222002+0000","last_undegraded":"2026-02-22T20:53:00.222002+0000","last_fullsized":"2026-02-22T20:53:00.222002+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":26193,"reported_epoch":43,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:53:02.289907+0000","last_change":"2026-02-22T20:53:00.228605+0000","last_active":"2026-02-22T20:53:02.289907+0000","last_peered":"2026-02-22T20:53:02.289907+0000","last_clean":"2026-02-22T20:50:28.894558+0000","last_became_active":"2026-02-22T20:51:26.182382+0000","last_became_peered":"2026-02-22T20:51:26.182382+0000","last_unstale":"2026-02-22T20:53:02.289907+0000","last_undegraded":"2026-02-22T20:53:02.289907+0000","last_fullsized":"2026-02-22T20:53:02.289907+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9906,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5286,"num_bytes_recovered":20569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10238}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":20482,"reported_epoch":44,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:53:07.289948+0000","last_change":"2026-02-22T20:53:06.816383+0000","last_active":"2026-02-22T20:53:07.289948+0000","last_peered":"2026-02-22T20:53:07.289948+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:07.289948+0000","last_undegraded":"2026-02-22T20:53:07.289948+0000","last_fullsized":"2026-02-22T20:53:07.289948+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13710,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3385,"num_bytes_recovered":3385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":23616,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68257,"num_bytes_recovered":297560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":988635136,"data_stored":1005489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80132,"ondisk_log_size":80132,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051606,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182424,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705909,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677816,"kb_used_data":289896,"kb_used_omap":25,"kb_used_meta":387878,"kb_avail":93694024,"statfs":{"total":96636764160,"available":95942680576,"internally_reserved":0,"allocated":296853504,"data_stored":436233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":397187256},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705910,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656076,"kb_used_data":286204,"kb_used_omap":25,"kb_used_meta":369830,"kb_avail":93715764,"statfs":{"total":96636764160,"available":95964942336,"internally_reserved":0,"allocated":293072896,"data_stored":435310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":378706104},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":296796160,"data_stored":301701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296792064,"data_stored":301700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311111680,"data_stored":305196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":83935232,"data_stored":96892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:11.941 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:12.130 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:12.130 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:12.143 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":645,"stamp":"2026-02-22T20:53:11.124376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13710,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73209,"num_bytes_recovered":302512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80094,"ondisk_log_size":80094,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1430768,"kb_used_data":618832,"kb_used_omap":54,"kb_used_meta":811849,"kb_avail":187312912,"statfs":{"total":193273528320,"available":191808421888,"internally_reserved":0,"allocated":633683968,"data_stored":956256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":55484,"internal_metadata":831334212},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10930,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5464,"num_bytes_recovered":5464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000863"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40908,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.235877+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:08.235877+0000","last_peered":"2026-02-22T20:53:08.235877+0000","last_clean":"2026-02-22T20:53:08.235877+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:08.235877+0000","last_undegraded":"2026-02-22T20:53:08.235877+0000","last_fullsized":"2026-02-22T20:53:08.235877+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41265,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.236537+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:08.236537+0000","last_peered":"2026-02-22T20:53:08.236537+0000","last_clean":"2026-02-22T20:53:08.236537+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:08.236537+0000","last_undegraded":"2026-02-22T20:53:08.236537+0000","last_fullsized":"2026-02-22T20:53:08.236537+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40510,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.236541+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:08.236541+0000","last_peered":"2026-02-22T20:53:08.236541+0000","last_clean":"2026-02-22T20:53:08.236541+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:08.236541+0000","last_undegraded":"2026-02-22T20:53:08.236541+0000","last_fullsized":"2026-02-22T20:53:08.236541+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41829,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.236574+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:08.236574+0000","last_peered":"2026-02-22T20:53:08.236574+0000","last_clean":"2026-02-22T20:53:08.236574+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:08.236574+0000","last_undegraded":"2026-02-22T20:53:08.236574+0000","last_fullsized":"2026-02-22T20:53:08.236574+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10131,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.235878+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:08.235878+0000","last_peered":"2026-02-22T20:53:08.235878+0000","last_clean":"2026-02-22T20:53:08.235878+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:08.235878+0000","last_undegraded":"2026-02-22T20:53:08.235878+0000","last_fullsized":"2026-02-22T20:53:08.235878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40770,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.236512+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:08.236512+0000","last_peered":"2026-02-22T20:53:08.236512+0000","last_clean":"2026-02-22T20:53:08.236512+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:08.236512+0000","last_undegraded":"2026-02-22T20:53:08.236512+0000","last_fullsized":"2026-02-22T20:53:08.236512+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41061,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.241395+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:08.241395+0000","last_peered":"2026-02-22T20:53:08.241395+0000","last_clean":"2026-02-22T20:53:08.241395+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:08.241395+0000","last_undegraded":"2026-02-22T20:53:08.241395+0000","last_fullsized":"2026-02-22T20:53:08.241395+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":20482,"reported_epoch":44,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:53:07.289948+0000","last_change":"2026-02-22T20:53:06.816383+0000","last_active":"2026-02-22T20:53:07.289948+0000","last_peered":"2026-02-22T20:53:07.289948+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:07.289948+0000","last_undegraded":"2026-02-22T20:53:07.289948+0000","last_fullsized":"2026-02-22T20:53:07.289948+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13710,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3385,"num_bytes_recovered":3385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13710,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73209,"num_bytes_recovered":302512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1032802304,"data_stored":1077392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80094,"ondisk_log_size":80094,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051606,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182424,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705910,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717428,"kb_used_data":311460,"kb_used_omap":27,"kb_used_meta":405924,"kb_avail":93654412,"statfs":{"total":96636764160,"available":95902117888,"internally_reserved":0,"allocated":318935040,"data_stored":478639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":415667106},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713340,"kb_used_data":307372,"kb_used_omap":27,"kb_used_meta":405924,"kb_avail":93658500,"statfs":{"total":96636764160,"available":95906304000,"internally_reserved":0,"allocated":314748928,"data_stored":477617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":415667106},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":318877696,"data_stored":337652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318877696,"data_stored":337652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311111680,"data_stored":305196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":83935232,"data_stored":96892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:12.144 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:12.335 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:12.335 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:12.350 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":645,"stamp":"2026-02-22T20:53:11.124376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13710,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73209,"num_bytes_recovered":302512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80094,"ondisk_log_size":80094,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1430768,"kb_used_data":618832,"kb_used_omap":54,"kb_used_meta":811849,"kb_avail":187312912,"statfs":{"total":193273528320,"available":191808421888,"internally_reserved":0,"allocated":633683968,"data_stored":956256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":55484,"internal_metadata":831334212},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10930,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5464,"num_bytes_recovered":5464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000863"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40908,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.235877+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:08.235877+0000","last_peered":"2026-02-22T20:53:08.235877+0000","last_clean":"2026-02-22T20:53:08.235877+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:08.235877+0000","last_undegraded":"2026-02-22T20:53:08.235877+0000","last_fullsized":"2026-02-22T20:53:08.235877+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41265,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.236537+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:08.236537+0000","last_peered":"2026-02-22T20:53:08.236537+0000","last_clean":"2026-02-22T20:53:08.236537+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:08.236537+0000","last_undegraded":"2026-02-22T20:53:08.236537+0000","last_fullsized":"2026-02-22T20:53:08.236537+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40510,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.236541+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:08.236541+0000","last_peered":"2026-02-22T20:53:08.236541+0000","last_clean":"2026-02-22T20:53:08.236541+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:08.236541+0000","last_undegraded":"2026-02-22T20:53:08.236541+0000","last_fullsized":"2026-02-22T20:53:08.236541+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41829,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.236574+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:08.236574+0000","last_peered":"2026-02-22T20:53:08.236574+0000","last_clean":"2026-02-22T20:53:08.236574+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:08.236574+0000","last_undegraded":"2026-02-22T20:53:08.236574+0000","last_fullsized":"2026-02-22T20:53:08.236574+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10131,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.235878+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:08.235878+0000","last_peered":"2026-02-22T20:53:08.235878+0000","last_clean":"2026-02-22T20:53:08.235878+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:08.235878+0000","last_undegraded":"2026-02-22T20:53:08.235878+0000","last_fullsized":"2026-02-22T20:53:08.235878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40770,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.236512+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:08.236512+0000","last_peered":"2026-02-22T20:53:08.236512+0000","last_clean":"2026-02-22T20:53:08.236512+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:08.236512+0000","last_undegraded":"2026-02-22T20:53:08.236512+0000","last_fullsized":"2026-02-22T20:53:08.236512+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41061,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.241395+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:08.241395+0000","last_peered":"2026-02-22T20:53:08.241395+0000","last_clean":"2026-02-22T20:53:08.241395+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:08.241395+0000","last_undegraded":"2026-02-22T20:53:08.241395+0000","last_fullsized":"2026-02-22T20:53:08.241395+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":20482,"reported_epoch":44,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:53:07.289948+0000","last_change":"2026-02-22T20:53:06.816383+0000","last_active":"2026-02-22T20:53:07.289948+0000","last_peered":"2026-02-22T20:53:07.289948+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:07.289948+0000","last_undegraded":"2026-02-22T20:53:07.289948+0000","last_fullsized":"2026-02-22T20:53:07.289948+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13710,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3385,"num_bytes_recovered":3385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13710,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73209,"num_bytes_recovered":302512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1032802304,"data_stored":1077392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80094,"ondisk_log_size":80094,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051606,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182424,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705910,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717428,"kb_used_data":311460,"kb_used_omap":27,"kb_used_meta":405924,"kb_avail":93654412,"statfs":{"total":96636764160,"available":95902117888,"internally_reserved":0,"allocated":318935040,"data_stored":478639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":415667106},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713340,"kb_used_data":307372,"kb_used_omap":27,"kb_used_meta":405924,"kb_avail":93658500,"statfs":{"total":96636764160,"available":95906304000,"internally_reserved":0,"allocated":314748928,"data_stored":477617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":415667106},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":318877696,"data_stored":337652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318877696,"data_stored":337652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311111680,"data_stored":305196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":83935232,"data_stored":96892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:12.351 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:53:12.351 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 20482, 'reported_epoch': 44, 'state': 'active+remapped+backfilling', 'last_fresh': '2026-02-22T20:53:07.289948+0000', 'last_change': '2026-02-22T20:53:06.816383+0000', 'last_active': '2026-02-22T20:53:07.289948+0000', 'last_peered': '2026-02-22T20:53:07.289948+0000', 'last_clean': '2026-02-22T20:50:31.380646+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:53:07.289948+0000', 'last_undegraded': '2026-02-22T20:53:07.289948+0000', 'last_fullsized': '2026-02-22T20:53:07.289948+0000', 'mapping_epoch': 30, 'log_start': "17'200", 'ondisk_log_start': "17'200", 'created': 10, 'last_epoch_clean': 16, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 13710, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3385, 'num_bytes_recovered': 3385, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [{'shards': '2,3', 'objects': 10239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2026-02-22T20:53:12.351 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:53:12.583 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:12.584 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1209,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":45,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":7},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":1465106432,"bytes_avail":191808421888,"bytes_total":193273528320,"misplaced_objects":13710,"misplaced_total":163508,"misplaced_ratio":0.083849108300511302,"recovering_objects_per_sec":682,"recovering_bytes_per_sec":682,"recovering_keys_per_sec":0,"num_objects_recovered":5464,"num_bytes_recovered":5464,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:53:12.598 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:53:12.599 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:12.788 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:12.788 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:12.802 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":645,"stamp":"2026-02-22T20:53:11.124376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13710,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73209,"num_bytes_recovered":302512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80094,"ondisk_log_size":80094,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1430768,"kb_used_data":618832,"kb_used_omap":54,"kb_used_meta":811849,"kb_avail":187312912,"statfs":{"total":193273528320,"available":191808421888,"internally_reserved":0,"allocated":633683968,"data_stored":956256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":55484,"internal_metadata":831334212},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-10930,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5464,"num_bytes_recovered":5464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000863"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40908,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.235877+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:08.235877+0000","last_peered":"2026-02-22T20:53:08.235877+0000","last_clean":"2026-02-22T20:53:08.235877+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:08.235877+0000","last_undegraded":"2026-02-22T20:53:08.235877+0000","last_fullsized":"2026-02-22T20:53:08.235877+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41265,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.236537+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:08.236537+0000","last_peered":"2026-02-22T20:53:08.236537+0000","last_clean":"2026-02-22T20:53:08.236537+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:08.236537+0000","last_undegraded":"2026-02-22T20:53:08.236537+0000","last_fullsized":"2026-02-22T20:53:08.236537+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40510,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.236541+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:08.236541+0000","last_peered":"2026-02-22T20:53:08.236541+0000","last_clean":"2026-02-22T20:53:08.236541+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:08.236541+0000","last_undegraded":"2026-02-22T20:53:08.236541+0000","last_fullsized":"2026-02-22T20:53:08.236541+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41829,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.236574+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:08.236574+0000","last_peered":"2026-02-22T20:53:08.236574+0000","last_clean":"2026-02-22T20:53:08.236574+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:08.236574+0000","last_undegraded":"2026-02-22T20:53:08.236574+0000","last_fullsized":"2026-02-22T20:53:08.236574+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10131,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.235878+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:08.235878+0000","last_peered":"2026-02-22T20:53:08.235878+0000","last_clean":"2026-02-22T20:53:08.235878+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:08.235878+0000","last_undegraded":"2026-02-22T20:53:08.235878+0000","last_fullsized":"2026-02-22T20:53:08.235878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40770,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.236512+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:08.236512+0000","last_peered":"2026-02-22T20:53:08.236512+0000","last_clean":"2026-02-22T20:53:08.236512+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:08.236512+0000","last_undegraded":"2026-02-22T20:53:08.236512+0000","last_fullsized":"2026-02-22T20:53:08.236512+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41061,"reported_epoch":45,"state":"active+clean","last_fresh":"2026-02-22T20:53:08.241395+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:08.241395+0000","last_peered":"2026-02-22T20:53:08.241395+0000","last_clean":"2026-02-22T20:53:08.241395+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:08.241395+0000","last_undegraded":"2026-02-22T20:53:08.241395+0000","last_fullsized":"2026-02-22T20:53:08.241395+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":20482,"reported_epoch":44,"state":"active+remapped+backfilling","last_fresh":"2026-02-22T20:53:07.289948+0000","last_change":"2026-02-22T20:53:06.816383+0000","last_active":"2026-02-22T20:53:07.289948+0000","last_peered":"2026-02-22T20:53:07.289948+0000","last_clean":"2026-02-22T20:50:31.380646+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:07.289948+0000","last_undegraded":"2026-02-22T20:53:07.289948+0000","last_fullsized":"2026-02-22T20:53:07.289948+0000","mapping_epoch":30,"log_start":"17'200","ondisk_log_start":"17'200","created":10,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13710,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3385,"num_bytes_recovered":3385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13710,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73209,"num_bytes_recovered":302512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1032802304,"data_stored":1077392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80094,"ondisk_log_size":80094,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051606,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182424,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705910,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717428,"kb_used_data":311460,"kb_used_omap":27,"kb_used_meta":405924,"kb_avail":93654412,"statfs":{"total":96636764160,"available":95902117888,"internally_reserved":0,"allocated":318935040,"data_stored":478639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":415667106},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713340,"kb_used_data":307372,"kb_used_omap":27,"kb_used_meta":405924,"kb_avail":93658500,"statfs":{"total":96636764160,"available":95906304000,"internally_reserved":0,"allocated":314748928,"data_stored":477617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":415667106},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":318877696,"data_stored":337652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318877696,"data_stored":337652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311111680,"data_stored":305196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":83935232,"data_stored":96892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:15.804 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:15.988 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:15.988 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:15.999 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":649,"stamp":"2026-02-22T20:53:15.251372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1539352,"kb_used_data":655224,"kb_used_omap":54,"kb_used_meta":884041,"kb_avail":187204328,"statfs":{"total":193273528320,"available":191697231872,"internally_reserved":0,"allocated":670949376,"data_stored":1029683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56135,"internal_metadata":905258169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-23616,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11806,"num_bytes_recovered":72938,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.016379"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40911,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.248933+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:14.248933+0000","last_peered":"2026-02-22T20:53:14.248933+0000","last_clean":"2026-02-22T20:53:14.248933+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:14.248933+0000","last_undegraded":"2026-02-22T20:53:14.248933+0000","last_fullsized":"2026-02-22T20:53:14.248933+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41268,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.250068+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:14.250068+0000","last_peered":"2026-02-22T20:53:14.250068+0000","last_clean":"2026-02-22T20:53:14.250068+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:14.250068+0000","last_undegraded":"2026-02-22T20:53:14.250068+0000","last_fullsized":"2026-02-22T20:53:14.250068+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40512,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.250016+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:14.250016+0000","last_peered":"2026-02-22T20:53:14.250016+0000","last_clean":"2026-02-22T20:53:14.250016+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:14.250016+0000","last_undegraded":"2026-02-22T20:53:14.250016+0000","last_fullsized":"2026-02-22T20:53:14.250016+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41831,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249977+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:14.249977+0000","last_peered":"2026-02-22T20:53:14.249977+0000","last_clean":"2026-02-22T20:53:14.249977+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:14.249977+0000","last_undegraded":"2026-02-22T20:53:14.249977+0000","last_fullsized":"2026-02-22T20:53:14.249977+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10133,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.248781+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:14.248781+0000","last_peered":"2026-02-22T20:53:14.248781+0000","last_clean":"2026-02-22T20:53:14.248781+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:14.248781+0000","last_undegraded":"2026-02-22T20:53:14.248781+0000","last_fullsized":"2026-02-22T20:53:14.248781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40772,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249984+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:14.249984+0000","last_peered":"2026-02-22T20:53:14.249984+0000","last_clean":"2026-02-22T20:53:14.249984+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:14.249984+0000","last_undegraded":"2026-02-22T20:53:14.249984+0000","last_fullsized":"2026-02-22T20:53:14.249984+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41063,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249998+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:14.249998+0000","last_peered":"2026-02-22T20:53:14.249998+0000","last_clean":"2026-02-22T20:53:14.249998+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:14.249998+0000","last_undegraded":"2026-02-22T20:53:14.249998+0000","last_fullsized":"2026-02-22T20:53:14.249998+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":41054,"reported_epoch":46,"state":"peering","last_fresh":"2026-02-22T20:53:13.657966+0000","last_change":"2026-02-22T20:53:15.015681+0000","last_active":"2026-02-22T20:53:15.015681+0000","last_peered":"2026-02-22T20:53:13.657966+0000","last_clean":"2026-02-22T20:53:13.657966+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:13.657966+0000","last_undegraded":"2026-02-22T20:53:13.657966+0000","last_fullsized":"2026-02-22T20:53:13.657966+0000","mapping_epoch":46,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1018990592,"data_stored":1119860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051607,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182425,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769680,"kb_used_data":327616,"kb_used_omap":27,"kb_used_meta":442020,"kb_avail":93602160,"statfs":{"total":96636764160,"available":95848611840,"internally_reserved":0,"allocated":335478784,"data_stored":516447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":452628759},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705912,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769672,"kb_used_data":327608,"kb_used_omap":27,"kb_used_meta":442020,"kb_avail":93602168,"statfs":{"total":96636764160,"available":95848620032,"internally_reserved":0,"allocated":335470592,"data_stored":513236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":452629410},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":306814976,"data_stored":304147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:16.001 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:16.181 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:16.182 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:16.193 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":649,"stamp":"2026-02-22T20:53:15.251372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1539352,"kb_used_data":655224,"kb_used_omap":54,"kb_used_meta":884041,"kb_avail":187204328,"statfs":{"total":193273528320,"available":191697231872,"internally_reserved":0,"allocated":670949376,"data_stored":1029683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56135,"internal_metadata":905258169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-23616,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11806,"num_bytes_recovered":72938,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.016379"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40911,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.248933+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:14.248933+0000","last_peered":"2026-02-22T20:53:14.248933+0000","last_clean":"2026-02-22T20:53:14.248933+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:14.248933+0000","last_undegraded":"2026-02-22T20:53:14.248933+0000","last_fullsized":"2026-02-22T20:53:14.248933+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41268,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.250068+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:14.250068+0000","last_peered":"2026-02-22T20:53:14.250068+0000","last_clean":"2026-02-22T20:53:14.250068+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:14.250068+0000","last_undegraded":"2026-02-22T20:53:14.250068+0000","last_fullsized":"2026-02-22T20:53:14.250068+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40512,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.250016+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:14.250016+0000","last_peered":"2026-02-22T20:53:14.250016+0000","last_clean":"2026-02-22T20:53:14.250016+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:14.250016+0000","last_undegraded":"2026-02-22T20:53:14.250016+0000","last_fullsized":"2026-02-22T20:53:14.250016+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41831,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249977+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:14.249977+0000","last_peered":"2026-02-22T20:53:14.249977+0000","last_clean":"2026-02-22T20:53:14.249977+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:14.249977+0000","last_undegraded":"2026-02-22T20:53:14.249977+0000","last_fullsized":"2026-02-22T20:53:14.249977+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10133,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.248781+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:14.248781+0000","last_peered":"2026-02-22T20:53:14.248781+0000","last_clean":"2026-02-22T20:53:14.248781+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:14.248781+0000","last_undegraded":"2026-02-22T20:53:14.248781+0000","last_fullsized":"2026-02-22T20:53:14.248781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40772,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249984+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:14.249984+0000","last_peered":"2026-02-22T20:53:14.249984+0000","last_clean":"2026-02-22T20:53:14.249984+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:14.249984+0000","last_undegraded":"2026-02-22T20:53:14.249984+0000","last_fullsized":"2026-02-22T20:53:14.249984+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41063,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249998+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:14.249998+0000","last_peered":"2026-02-22T20:53:14.249998+0000","last_clean":"2026-02-22T20:53:14.249998+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:14.249998+0000","last_undegraded":"2026-02-22T20:53:14.249998+0000","last_fullsized":"2026-02-22T20:53:14.249998+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":41054,"reported_epoch":46,"state":"peering","last_fresh":"2026-02-22T20:53:13.657966+0000","last_change":"2026-02-22T20:53:15.015681+0000","last_active":"2026-02-22T20:53:15.015681+0000","last_peered":"2026-02-22T20:53:13.657966+0000","last_clean":"2026-02-22T20:53:13.657966+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:13.657966+0000","last_undegraded":"2026-02-22T20:53:13.657966+0000","last_fullsized":"2026-02-22T20:53:13.657966+0000","mapping_epoch":46,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1018990592,"data_stored":1119860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051607,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182425,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769680,"kb_used_data":327616,"kb_used_omap":27,"kb_used_meta":442020,"kb_avail":93602160,"statfs":{"total":96636764160,"available":95848611840,"internally_reserved":0,"allocated":335478784,"data_stored":516447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":452628759},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705912,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769672,"kb_used_data":327608,"kb_used_omap":27,"kb_used_meta":442020,"kb_avail":93602168,"statfs":{"total":96636764160,"available":95848620032,"internally_reserved":0,"allocated":335470592,"data_stored":513236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":452629410},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":306814976,"data_stored":304147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:16.194 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:53:16.194 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 41054, 'reported_epoch': 46, 'state': 'peering', 'last_fresh': '2026-02-22T20:53:13.657966+0000', 'last_change': '2026-02-22T20:53:15.015681+0000', 'last_active': '2026-02-22T20:53:15.015681+0000', 'last_peered': '2026-02-22T20:53:13.657966+0000', 'last_clean': '2026-02-22T20:53:13.657966+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:53:13.657966+0000', 'last_undegraded': '2026-02-22T20:53:13.657966+0000', 'last_fullsized': '2026-02-22T20:53:13.657966+0000', 'mapping_epoch': 46, 'log_start': "17'239", 'ondisk_log_start': "17'239", 'created': 10, 'last_epoch_clean': 45, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10000, 'log_dups_size': 0, 'ondisk_log_size': 10000, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10239, 'num_bytes_recovered': 71371, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2026-02-22T20:53:16.195 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:53:16.412 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:16.412 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1213,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":47,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":7},{"state_name":"peering","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":1576296448,"bytes_avail":191697231872,"bytes_total":193273528320,"inactive_pgs_ratio":0.125,"recovering_objects_per_sec":1560,"recovering_bytes_per_sec":9307,"recovering_keys_per_sec":0,"num_objects_recovered":12318,"num_bytes_recovered":73450,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:53:16.423 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:53:16.423 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:16.597 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:16.597 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:16.608 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":649,"stamp":"2026-02-22T20:53:15.251372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1539352,"kb_used_data":655224,"kb_used_omap":54,"kb_used_meta":884041,"kb_avail":187204328,"statfs":{"total":193273528320,"available":191697231872,"internally_reserved":0,"allocated":670949376,"data_stored":1029683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56135,"internal_metadata":905258169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-23616,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11806,"num_bytes_recovered":72938,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.016379"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40911,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.248933+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:14.248933+0000","last_peered":"2026-02-22T20:53:14.248933+0000","last_clean":"2026-02-22T20:53:14.248933+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:14.248933+0000","last_undegraded":"2026-02-22T20:53:14.248933+0000","last_fullsized":"2026-02-22T20:53:14.248933+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41268,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.250068+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:14.250068+0000","last_peered":"2026-02-22T20:53:14.250068+0000","last_clean":"2026-02-22T20:53:14.250068+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:14.250068+0000","last_undegraded":"2026-02-22T20:53:14.250068+0000","last_fullsized":"2026-02-22T20:53:14.250068+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40512,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.250016+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:14.250016+0000","last_peered":"2026-02-22T20:53:14.250016+0000","last_clean":"2026-02-22T20:53:14.250016+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:14.250016+0000","last_undegraded":"2026-02-22T20:53:14.250016+0000","last_fullsized":"2026-02-22T20:53:14.250016+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41831,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249977+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:14.249977+0000","last_peered":"2026-02-22T20:53:14.249977+0000","last_clean":"2026-02-22T20:53:14.249977+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:14.249977+0000","last_undegraded":"2026-02-22T20:53:14.249977+0000","last_fullsized":"2026-02-22T20:53:14.249977+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10133,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.248781+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:14.248781+0000","last_peered":"2026-02-22T20:53:14.248781+0000","last_clean":"2026-02-22T20:53:14.248781+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:14.248781+0000","last_undegraded":"2026-02-22T20:53:14.248781+0000","last_fullsized":"2026-02-22T20:53:14.248781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40772,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249984+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:14.249984+0000","last_peered":"2026-02-22T20:53:14.249984+0000","last_clean":"2026-02-22T20:53:14.249984+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:14.249984+0000","last_undegraded":"2026-02-22T20:53:14.249984+0000","last_fullsized":"2026-02-22T20:53:14.249984+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41063,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249998+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:14.249998+0000","last_peered":"2026-02-22T20:53:14.249998+0000","last_clean":"2026-02-22T20:53:14.249998+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:14.249998+0000","last_undegraded":"2026-02-22T20:53:14.249998+0000","last_fullsized":"2026-02-22T20:53:14.249998+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":41054,"reported_epoch":46,"state":"peering","last_fresh":"2026-02-22T20:53:13.657966+0000","last_change":"2026-02-22T20:53:15.015681+0000","last_active":"2026-02-22T20:53:15.015681+0000","last_peered":"2026-02-22T20:53:13.657966+0000","last_clean":"2026-02-22T20:53:13.657966+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:13.657966+0000","last_undegraded":"2026-02-22T20:53:13.657966+0000","last_fullsized":"2026-02-22T20:53:13.657966+0000","mapping_epoch":46,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1018990592,"data_stored":1119860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051607,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182425,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769680,"kb_used_data":327616,"kb_used_omap":27,"kb_used_meta":442020,"kb_avail":93602160,"statfs":{"total":96636764160,"available":95848611840,"internally_reserved":0,"allocated":335478784,"data_stored":516447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":452628759},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705912,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769672,"kb_used_data":327608,"kb_used_omap":27,"kb_used_meta":442020,"kb_avail":93602168,"statfs":{"total":96636764160,"available":95848620032,"internally_reserved":0,"allocated":335470592,"data_stored":513236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":452629410},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":306814976,"data_stored":304147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:19.609 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:19.790 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:19.790 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:19.802 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":651,"stamp":"2026-02-22T20:53:19.125275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1539352,"kb_used_data":655224,"kb_used_omap":54,"kb_used_meta":884041,"kb_avail":187204328,"statfs":{"total":193273528320,"available":191697231872,"internally_reserved":0,"allocated":670949376,"data_stored":1029683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56135,"internal_metadata":905258169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-13710,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6854,"num_bytes_recovered":67986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000899"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40911,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.248933+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:14.248933+0000","last_peered":"2026-02-22T20:53:14.248933+0000","last_clean":"2026-02-22T20:53:14.248933+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:14.248933+0000","last_undegraded":"2026-02-22T20:53:14.248933+0000","last_fullsized":"2026-02-22T20:53:14.248933+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41268,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.250068+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:14.250068+0000","last_peered":"2026-02-22T20:53:14.250068+0000","last_clean":"2026-02-22T20:53:14.250068+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:14.250068+0000","last_undegraded":"2026-02-22T20:53:14.250068+0000","last_fullsized":"2026-02-22T20:53:14.250068+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40512,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.250016+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:14.250016+0000","last_peered":"2026-02-22T20:53:14.250016+0000","last_clean":"2026-02-22T20:53:14.250016+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:14.250016+0000","last_undegraded":"2026-02-22T20:53:14.250016+0000","last_fullsized":"2026-02-22T20:53:14.250016+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41831,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249977+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:14.249977+0000","last_peered":"2026-02-22T20:53:14.249977+0000","last_clean":"2026-02-22T20:53:14.249977+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:14.249977+0000","last_undegraded":"2026-02-22T20:53:14.249977+0000","last_fullsized":"2026-02-22T20:53:14.249977+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10133,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.248781+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:14.248781+0000","last_peered":"2026-02-22T20:53:14.248781+0000","last_clean":"2026-02-22T20:53:14.248781+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:14.248781+0000","last_undegraded":"2026-02-22T20:53:14.248781+0000","last_fullsized":"2026-02-22T20:53:14.248781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40772,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249984+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:14.249984+0000","last_peered":"2026-02-22T20:53:14.249984+0000","last_clean":"2026-02-22T20:53:14.249984+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:14.249984+0000","last_undegraded":"2026-02-22T20:53:14.249984+0000","last_fullsized":"2026-02-22T20:53:14.249984+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41063,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249998+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:14.249998+0000","last_peered":"2026-02-22T20:53:14.249998+0000","last_clean":"2026-02-22T20:53:14.249998+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:14.249998+0000","last_undegraded":"2026-02-22T20:53:14.249998+0000","last_fullsized":"2026-02-22T20:53:14.249998+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":41054,"reported_epoch":46,"state":"peering","last_fresh":"2026-02-22T20:53:13.657966+0000","last_change":"2026-02-22T20:53:15.015681+0000","last_active":"2026-02-22T20:53:15.015681+0000","last_peered":"2026-02-22T20:53:13.657966+0000","last_clean":"2026-02-22T20:53:13.657966+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:13.657966+0000","last_undegraded":"2026-02-22T20:53:13.657966+0000","last_fullsized":"2026-02-22T20:53:13.657966+0000","mapping_epoch":46,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1014325248,"data_stored":1118721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051608,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182426,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769680,"kb_used_data":327616,"kb_used_omap":27,"kb_used_meta":442020,"kb_avail":93602160,"statfs":{"total":96636764160,"available":95848611840,"internally_reserved":0,"allocated":335478784,"data_stored":516447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":452628759},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705912,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769672,"kb_used_data":327608,"kb_used_omap":27,"kb_used_meta":442020,"kb_avail":93602168,"statfs":{"total":96636764160,"available":95848620032,"internally_reserved":0,"allocated":335470592,"data_stored":513236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":452629410},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302149632,"data_stored":303008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:19.802 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:19.977 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:19.977 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:19.989 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":651,"stamp":"2026-02-22T20:53:19.125275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1539352,"kb_used_data":655224,"kb_used_omap":54,"kb_used_meta":884041,"kb_avail":187204328,"statfs":{"total":193273528320,"available":191697231872,"internally_reserved":0,"allocated":670949376,"data_stored":1029683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56135,"internal_metadata":905258169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-13710,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6854,"num_bytes_recovered":67986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000899"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40911,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.248933+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:14.248933+0000","last_peered":"2026-02-22T20:53:14.248933+0000","last_clean":"2026-02-22T20:53:14.248933+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:14.248933+0000","last_undegraded":"2026-02-22T20:53:14.248933+0000","last_fullsized":"2026-02-22T20:53:14.248933+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41268,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.250068+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:14.250068+0000","last_peered":"2026-02-22T20:53:14.250068+0000","last_clean":"2026-02-22T20:53:14.250068+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:14.250068+0000","last_undegraded":"2026-02-22T20:53:14.250068+0000","last_fullsized":"2026-02-22T20:53:14.250068+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40512,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.250016+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:14.250016+0000","last_peered":"2026-02-22T20:53:14.250016+0000","last_clean":"2026-02-22T20:53:14.250016+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:14.250016+0000","last_undegraded":"2026-02-22T20:53:14.250016+0000","last_fullsized":"2026-02-22T20:53:14.250016+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41831,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249977+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:14.249977+0000","last_peered":"2026-02-22T20:53:14.249977+0000","last_clean":"2026-02-22T20:53:14.249977+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:14.249977+0000","last_undegraded":"2026-02-22T20:53:14.249977+0000","last_fullsized":"2026-02-22T20:53:14.249977+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10133,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.248781+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:14.248781+0000","last_peered":"2026-02-22T20:53:14.248781+0000","last_clean":"2026-02-22T20:53:14.248781+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:14.248781+0000","last_undegraded":"2026-02-22T20:53:14.248781+0000","last_fullsized":"2026-02-22T20:53:14.248781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40772,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249984+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:14.249984+0000","last_peered":"2026-02-22T20:53:14.249984+0000","last_clean":"2026-02-22T20:53:14.249984+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:14.249984+0000","last_undegraded":"2026-02-22T20:53:14.249984+0000","last_fullsized":"2026-02-22T20:53:14.249984+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41063,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249998+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:14.249998+0000","last_peered":"2026-02-22T20:53:14.249998+0000","last_clean":"2026-02-22T20:53:14.249998+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:14.249998+0000","last_undegraded":"2026-02-22T20:53:14.249998+0000","last_fullsized":"2026-02-22T20:53:14.249998+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":41054,"reported_epoch":46,"state":"peering","last_fresh":"2026-02-22T20:53:13.657966+0000","last_change":"2026-02-22T20:53:15.015681+0000","last_active":"2026-02-22T20:53:15.015681+0000","last_peered":"2026-02-22T20:53:13.657966+0000","last_clean":"2026-02-22T20:53:13.657966+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:13.657966+0000","last_undegraded":"2026-02-22T20:53:13.657966+0000","last_fullsized":"2026-02-22T20:53:13.657966+0000","mapping_epoch":46,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1014325248,"data_stored":1118721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051608,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182426,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769680,"kb_used_data":327616,"kb_used_omap":27,"kb_used_meta":442020,"kb_avail":93602160,"statfs":{"total":96636764160,"available":95848611840,"internally_reserved":0,"allocated":335478784,"data_stored":516447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":452628759},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705912,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769672,"kb_used_data":327608,"kb_used_omap":27,"kb_used_meta":442020,"kb_avail":93602168,"statfs":{"total":96636764160,"available":95848620032,"internally_reserved":0,"allocated":335470592,"data_stored":513236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":452629410},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302149632,"data_stored":303008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:19.991 INFO:tasks.osd_recovery.ceph_manager:PG 1.0 is not active+clean 2026-02-22T20:53:19.992 INFO:tasks.osd_recovery.ceph_manager:{'pgid': '1.0', 'version': "17'10239", 'reported_seq': 41054, 'reported_epoch': 46, 'state': 'peering', 'last_fresh': '2026-02-22T20:53:13.657966+0000', 'last_change': '2026-02-22T20:53:15.015681+0000', 'last_active': '2026-02-22T20:53:15.015681+0000', 'last_peered': '2026-02-22T20:53:13.657966+0000', 'last_clean': '2026-02-22T20:53:13.657966+0000', 'last_became_active': '2026-02-22T20:51:26.182466+0000', 'last_became_peered': '2026-02-22T20:51:26.182466+0000', 'last_unstale': '2026-02-22T20:53:13.657966+0000', 'last_undegraded': '2026-02-22T20:53:13.657966+0000', 'last_fullsized': '2026-02-22T20:53:13.657966+0000', 'mapping_epoch': 46, 'log_start': "17'239", 'ondisk_log_start': "17'239", 'created': 10, 'last_epoch_clean': 45, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'last_clean_scrub_stamp': '2026-02-22T20:33:12.997318+0000', 'objects_scrubbed': 0, 'log_size': 10000, 'log_dups_size': 0, 'ondisk_log_size': 10000, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 71371, 'num_objects': 10239, 'num_object_clones': 0, 'num_object_copies': 20478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 10295, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10239, 'num_bytes_recovered': 71371, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2026-02-22T20:53:19.992 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-22T20:53:20.206 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:20.206 INFO:teuthology.orchestra.run.trial084.stdout:{"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1216,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":3},"osdmap":{"epoch":47,"num_osds":4,"num_up_osds":4,"osd_up_since":1771793484,"num_in_osds":2,"osd_in_since":1771793433,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":7},{"state_name":"peering","count":1}],"num_pgs":8,"num_pools":1,"num_objects":81754,"data_bytes":372171,"bytes_used":1576296448,"bytes_avail":191697231872,"bytes_total":193273528320,"inactive_pgs_ratio":0.125,"recovering_objects_per_sec":856,"recovering_bytes_per_sec":8497,"recovering_keys_per_sec":0,"num_objects_recovered":6854,"num_bytes_recovered":67986,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2026-02-22T20:33:03:266748+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":7,"modified":"2026-02-22T20:52:25.119246+0000","services":{}},"progress_events":{}} 2026-02-22T20:53:20.218 INFO:tasks.osd_recovery.ceph_manager:making progress, resetting timeout 2026-02-22T20:53:20.219 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:20.398 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:20.398 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:20.410 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":651,"stamp":"2026-02-22T20:53:19.125275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1539352,"kb_used_data":655224,"kb_used_omap":54,"kb_used_meta":884041,"kb_avail":187204328,"statfs":{"total":193273528320,"available":191697231872,"internally_reserved":0,"allocated":670949376,"data_stored":1029683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56135,"internal_metadata":905258169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-13710,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6854,"num_bytes_recovered":67986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.000899"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40911,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.248933+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:14.248933+0000","last_peered":"2026-02-22T20:53:14.248933+0000","last_clean":"2026-02-22T20:53:14.248933+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:14.248933+0000","last_undegraded":"2026-02-22T20:53:14.248933+0000","last_fullsized":"2026-02-22T20:53:14.248933+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41268,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.250068+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:14.250068+0000","last_peered":"2026-02-22T20:53:14.250068+0000","last_clean":"2026-02-22T20:53:14.250068+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:14.250068+0000","last_undegraded":"2026-02-22T20:53:14.250068+0000","last_fullsized":"2026-02-22T20:53:14.250068+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40512,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.250016+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:14.250016+0000","last_peered":"2026-02-22T20:53:14.250016+0000","last_clean":"2026-02-22T20:53:14.250016+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:14.250016+0000","last_undegraded":"2026-02-22T20:53:14.250016+0000","last_fullsized":"2026-02-22T20:53:14.250016+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41831,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249977+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:14.249977+0000","last_peered":"2026-02-22T20:53:14.249977+0000","last_clean":"2026-02-22T20:53:14.249977+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:14.249977+0000","last_undegraded":"2026-02-22T20:53:14.249977+0000","last_fullsized":"2026-02-22T20:53:14.249977+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10133,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.248781+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:14.248781+0000","last_peered":"2026-02-22T20:53:14.248781+0000","last_clean":"2026-02-22T20:53:14.248781+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:14.248781+0000","last_undegraded":"2026-02-22T20:53:14.248781+0000","last_fullsized":"2026-02-22T20:53:14.248781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40772,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249984+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:14.249984+0000","last_peered":"2026-02-22T20:53:14.249984+0000","last_clean":"2026-02-22T20:53:14.249984+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:14.249984+0000","last_undegraded":"2026-02-22T20:53:14.249984+0000","last_fullsized":"2026-02-22T20:53:14.249984+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41063,"reported_epoch":46,"state":"active+clean","last_fresh":"2026-02-22T20:53:14.249998+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:14.249998+0000","last_peered":"2026-02-22T20:53:14.249998+0000","last_clean":"2026-02-22T20:53:14.249998+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:14.249998+0000","last_undegraded":"2026-02-22T20:53:14.249998+0000","last_fullsized":"2026-02-22T20:53:14.249998+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":41054,"reported_epoch":46,"state":"peering","last_fresh":"2026-02-22T20:53:13.657966+0000","last_change":"2026-02-22T20:53:15.015681+0000","last_active":"2026-02-22T20:53:15.015681+0000","last_peered":"2026-02-22T20:53:13.657966+0000","last_clean":"2026-02-22T20:53:13.657966+0000","last_became_active":"2026-02-22T20:51:26.182466+0000","last_became_peered":"2026-02-22T20:51:26.182466+0000","last_unstale":"2026-02-22T20:53:13.657966+0000","last_undegraded":"2026-02-22T20:53:13.657966+0000","last_fullsized":"2026-02-22T20:53:13.657966+0000","mapping_epoch":46,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1014325248,"data_stored":1118721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051608,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182426,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769680,"kb_used_data":327616,"kb_used_omap":27,"kb_used_meta":442020,"kb_avail":93602160,"statfs":{"total":96636764160,"available":95848611840,"internally_reserved":0,"allocated":335478784,"data_stored":516447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28393,"internal_metadata":452628759},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705912,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769672,"kb_used_data":327608,"kb_used_omap":27,"kb_used_meta":442020,"kb_avail":93602168,"statfs":{"total":96636764160,"available":95848620032,"internally_reserved":0,"allocated":335470592,"data_stored":513236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":452629410},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302149632,"data_stored":303008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:23.413 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:23.593 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:23.593 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:23.605 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":653,"stamp":"2026-02-22T20:53:23.125717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1539376,"kb_used_data":655248,"kb_used_omap":56,"kb_used_meta":884039,"kb_avail":187204304,"statfs":{"total":193273528320,"available":191697207296,"internally_reserved":0,"allocated":670973952,"data_stored":1039274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58088,"internal_metadata":905256216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2958,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1479,"num_bytes_recovered":1479,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.879158"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40913,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.252908+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:15.252908+0000","last_peered":"2026-02-22T20:53:15.252908+0000","last_clean":"2026-02-22T20:53:15.252908+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:15.252908+0000","last_undegraded":"2026-02-22T20:53:15.252908+0000","last_fullsized":"2026-02-22T20:53:15.252908+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41270,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253746+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:15.253746+0000","last_peered":"2026-02-22T20:53:15.253746+0000","last_clean":"2026-02-22T20:53:15.253746+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:15.253746+0000","last_undegraded":"2026-02-22T20:53:15.253746+0000","last_fullsized":"2026-02-22T20:53:15.253746+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40514,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253720+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:15.253720+0000","last_peered":"2026-02-22T20:53:15.253720+0000","last_clean":"2026-02-22T20:53:15.253720+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:15.253720+0000","last_undegraded":"2026-02-22T20:53:15.253720+0000","last_fullsized":"2026-02-22T20:53:15.253720+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41833,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253728+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:15.253728+0000","last_peered":"2026-02-22T20:53:15.253728+0000","last_clean":"2026-02-22T20:53:15.253728+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:15.253728+0000","last_undegraded":"2026-02-22T20:53:15.253728+0000","last_fullsized":"2026-02-22T20:53:15.253728+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10135,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.252866+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:15.252866+0000","last_peered":"2026-02-22T20:53:15.252866+0000","last_clean":"2026-02-22T20:53:15.252866+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:15.252866+0000","last_undegraded":"2026-02-22T20:53:15.252866+0000","last_fullsized":"2026-02-22T20:53:15.252866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40774,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253710+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:15.253710+0000","last_peered":"2026-02-22T20:53:15.253710+0000","last_clean":"2026-02-22T20:53:15.253710+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:15.253710+0000","last_undegraded":"2026-02-22T20:53:15.253710+0000","last_fullsized":"2026-02-22T20:53:15.253710+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41065,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253702+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:15.253702+0000","last_peered":"2026-02-22T20:53:15.253702+0000","last_clean":"2026-02-22T20:53:15.253702+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:15.253702+0000","last_undegraded":"2026-02-22T20:53:15.253702+0000","last_fullsized":"2026-02-22T20:53:15.253702+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":41063,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.258082+0000","last_change":"2026-02-22T20:53:15.258045+0000","last_active":"2026-02-22T20:53:15.258082+0000","last_peered":"2026-02-22T20:53:15.258082+0000","last_clean":"2026-02-22T20:53:15.258082+0000","last_became_active":"2026-02-22T20:53:15.254030+0000","last_became_peered":"2026-02-22T20:53:15.254030+0000","last_unstale":"2026-02-22T20:53:15.258082+0000","last_undegraded":"2026-02-22T20:53:15.258082+0000","last_fullsized":"2026-02-22T20:53:15.258082+0000","mapping_epoch":46,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:27:41.800790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1009410048,"data_stored":1117521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051608,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182427,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705912,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769688,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":442019,"kb_avail":93602152,"statfs":{"total":96636764160,"available":95848603648,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":452628108},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705913,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769688,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":442019,"kb_avail":93602152,"statfs":{"total":96636764160,"available":95848603648,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":452628108},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":297234432,"data_stored":301808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:23.606 INFO:tasks.osd_recovery.ceph_manager:clean! 2026-02-22T20:53:23.607 INFO:teuthology.run_tasks:Running task pexec... 2026-02-22T20:53:23.629 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-22T20:53:23.630 DEBUG:teuthology.orchestra.run.trial084:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-22T20:53:23.633 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial084.front.sepia.ceph.com 2026-02-22T20:53:23.633 INFO:teuthology.task.pexec:sudo dnf remove nvme-cli -y 2026-02-22T20:53:23.633 INFO:teuthology.task.pexec:sudo dnf install nvmetcli nvme-cli -y 2026-02-22T20:53:23.818 INFO:teuthology.orchestra.run.trial084.stdout:No match for argument: nvme-cli 2026-02-22T20:53:23.818 INFO:teuthology.orchestra.run.trial084.stderr:No packages marked for removal. 2026-02-22T20:53:23.821 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:53:23.821 INFO:teuthology.orchestra.run.trial084.stdout:Nothing to do. 2026-02-22T20:53:23.821 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:53:24.110 INFO:teuthology.orchestra.run.trial084.stdout:Last metadata expiration check: 0:21:36 ago on Sun 22 Feb 2026 08:31:48 PM UTC. 2026-02-22T20:53:24.150 INFO:teuthology.orchestra.run.trial084.stdout:Dependencies resolved. 2026-02-22T20:53:24.150 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout: Package Arch Version Repository Size 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout:Installing: 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout: nvme-cli x86_64 2.13-2.el10 baseos 1.1 M 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout: nvmetcli noarch 0.8-1.el10 baseos 54 k 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout:Installing dependencies: 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout: python3-configshell noarch 1:1.1.30-8.el10 baseos 90 k 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout: python3-kmod x86_64 0.9.2-6.el10 baseos 111 k 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout: python3-pyparsing noarch 3.1.1-7.el10 baseos 271 k 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout: python3-urwid x86_64 2.5.3-4.el10 baseos 1.1 M 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout:Transaction Summary 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout:================================================================================ 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout:Install 6 Packages 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout:Total download size: 2.7 M 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout:Installed size: 12 M 2026-02-22T20:53:24.151 INFO:teuthology.orchestra.run.trial084.stdout:Downloading Packages: 2026-02-22T20:53:24.196 INFO:teuthology.orchestra.run.trial084.stdout:(1/6): nvmetcli-0.8-1.el10.noarch.rpm 1.2 MB/s | 54 kB 00:00 2026-02-22T20:53:24.202 INFO:teuthology.orchestra.run.trial084.stdout:(2/6): python3-configshell-1.1.30-8.el10.noarch 1.8 MB/s | 90 kB 00:00 2026-02-22T20:53:24.211 INFO:teuthology.orchestra.run.trial084.stdout:(3/6): nvme-cli-2.13-2.el10.x86_64.rpm 19 MB/s | 1.1 MB 00:00 2026-02-22T20:53:24.212 INFO:teuthology.orchestra.run.trial084.stdout:(4/6): python3-kmod-0.9.2-6.el10.x86_64.rpm 6.7 MB/s | 111 kB 00:00 2026-02-22T20:53:24.218 INFO:teuthology.orchestra.run.trial084.stdout:(5/6): python3-pyparsing-3.1.1-7.el10.noarch.rp 17 MB/s | 271 kB 00:00 2026-02-22T20:53:24.237 INFO:teuthology.orchestra.run.trial084.stdout:(6/6): python3-urwid-2.5.3-4.el10.x86_64.rpm 40 MB/s | 1.1 MB 00:00 2026-02-22T20:53:24.238 INFO:teuthology.orchestra.run.trial084.stdout:-------------------------------------------------------------------------------- 2026-02-22T20:53:24.238 INFO:teuthology.orchestra.run.trial084.stdout:Total 31 MB/s | 2.7 MB 00:00 2026-02-22T20:53:24.238 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction check 2026-02-22T20:53:24.246 INFO:teuthology.orchestra.run.trial084.stdout:Transaction check succeeded. 2026-02-22T20:53:24.246 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction test 2026-02-22T20:53:24.301 INFO:teuthology.orchestra.run.trial084.stdout:Transaction test succeeded. 2026-02-22T20:53:24.302 INFO:teuthology.orchestra.run.trial084.stdout:Running transaction 2026-02-22T20:53:24.396 INFO:teuthology.orchestra.run.trial084.stdout: Preparing : 1/1 2026-02-22T20:53:24.404 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-urwid-2.5.3-4.el10.x86_64 1/6 2026-02-22T20:53:24.408 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-pyparsing-3.1.1-7.el10.noarch 2/6 2026-02-22T20:53:24.411 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-configshell-1:1.1.30-8.el10.noarch 3/6 2026-02-22T20:53:24.415 INFO:teuthology.orchestra.run.trial084.stdout: Installing : python3-kmod-0.9.2-6.el10.x86_64 4/6 2026-02-22T20:53:24.416 INFO:teuthology.orchestra.run.trial084.stdout: Installing : nvmetcli-0.8-1.el10.noarch 5/6 2026-02-22T20:53:24.473 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: nvmetcli-0.8-1.el10.noarch 5/6 2026-02-22T20:53:24.475 INFO:teuthology.orchestra.run.trial084.stdout: Installing : nvme-cli-2.13-2.el10.x86_64 6/6 2026-02-22T20:53:24.745 INFO:teuthology.orchestra.run.trial084.stdout: Running scriptlet: nvme-cli-2.13-2.el10.x86_64 6/6 2026-02-22T20:53:24.745 INFO:teuthology.orchestra.run.trial084.stdout:Created symlink '/etc/systemd/system/default.target.wants/nvmefc-boot-connections.service' → '/usr/lib/systemd/system/nvmefc-boot-connections.service'. 2026-02-22T20:53:24.745 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:33.135 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:33.136 INFO:teuthology.orchestra.run.trial084.stdout:Installed: 2026-02-22T20:53:33.136 INFO:teuthology.orchestra.run.trial084.stdout: nvme-cli-2.13-2.el10.x86_64 nvmetcli-0.8-1.el10.noarch 2026-02-22T20:53:33.136 INFO:teuthology.orchestra.run.trial084.stdout: python3-configshell-1:1.1.30-8.el10.noarch python3-kmod-0.9.2-6.el10.x86_64 2026-02-22T20:53:33.136 INFO:teuthology.orchestra.run.trial084.stdout: python3-pyparsing-3.1.1-7.el10.noarch python3-urwid-2.5.3-4.el10.x86_64 2026-02-22T20:53:33.136 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:33.136 INFO:teuthology.orchestra.run.trial084.stdout:Complete! 2026-02-22T20:53:33.160 DEBUG:teuthology.parallel:result is None 2026-02-22T20:53:33.160 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-22T20:53:33.173 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-22T20:53:33.174 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:33.354 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:33.354 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:33.366 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":658,"stamp":"2026-02-22T20:53:33.126774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1539376,"kb_used_data":655248,"kb_used_omap":56,"kb_used_meta":884039,"kb_avail":187204304,"statfs":{"total":193273528320,"available":191697207296,"internally_reserved":0,"allocated":670973952,"data_stored":1039274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58088,"internal_metadata":905256216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001273"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40913,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.252908+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:15.252908+0000","last_peered":"2026-02-22T20:53:15.252908+0000","last_clean":"2026-02-22T20:53:15.252908+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:15.252908+0000","last_undegraded":"2026-02-22T20:53:15.252908+0000","last_fullsized":"2026-02-22T20:53:15.252908+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41270,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253746+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:15.253746+0000","last_peered":"2026-02-22T20:53:15.253746+0000","last_clean":"2026-02-22T20:53:15.253746+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:15.253746+0000","last_undegraded":"2026-02-22T20:53:15.253746+0000","last_fullsized":"2026-02-22T20:53:15.253746+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40514,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253720+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:15.253720+0000","last_peered":"2026-02-22T20:53:15.253720+0000","last_clean":"2026-02-22T20:53:15.253720+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:15.253720+0000","last_undegraded":"2026-02-22T20:53:15.253720+0000","last_fullsized":"2026-02-22T20:53:15.253720+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41833,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253728+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:15.253728+0000","last_peered":"2026-02-22T20:53:15.253728+0000","last_clean":"2026-02-22T20:53:15.253728+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:15.253728+0000","last_undegraded":"2026-02-22T20:53:15.253728+0000","last_fullsized":"2026-02-22T20:53:15.253728+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10135,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.252866+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:15.252866+0000","last_peered":"2026-02-22T20:53:15.252866+0000","last_clean":"2026-02-22T20:53:15.252866+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:15.252866+0000","last_undegraded":"2026-02-22T20:53:15.252866+0000","last_fullsized":"2026-02-22T20:53:15.252866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40774,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253710+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:15.253710+0000","last_peered":"2026-02-22T20:53:15.253710+0000","last_clean":"2026-02-22T20:53:15.253710+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:15.253710+0000","last_undegraded":"2026-02-22T20:53:15.253710+0000","last_fullsized":"2026-02-22T20:53:15.253710+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41065,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253702+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:15.253702+0000","last_peered":"2026-02-22T20:53:15.253702+0000","last_clean":"2026-02-22T20:53:15.253702+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:15.253702+0000","last_undegraded":"2026-02-22T20:53:15.253702+0000","last_fullsized":"2026-02-22T20:53:15.253702+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":41063,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.258082+0000","last_change":"2026-02-22T20:53:15.258045+0000","last_active":"2026-02-22T20:53:15.258082+0000","last_peered":"2026-02-22T20:53:15.258082+0000","last_clean":"2026-02-22T20:53:15.258082+0000","last_became_active":"2026-02-22T20:53:15.254030+0000","last_became_peered":"2026-02-22T20:53:15.254030+0000","last_unstale":"2026-02-22T20:53:15.258082+0000","last_undegraded":"2026-02-22T20:53:15.258082+0000","last_fullsized":"2026-02-22T20:53:15.258082+0000","mapping_epoch":46,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:27:41.800790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1000562688,"data_stored":1115361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051610,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182429,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705914,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769688,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":442019,"kb_avail":93602152,"statfs":{"total":96636764160,"available":95848603648,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":452628108},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705915,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769688,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":442019,"kb_avail":93602152,"statfs":{"total":96636764160,"available":95848603648,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":452628108},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288387072,"data_stored":299648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:33.367 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:33.549 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:33.550 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:33.561 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":658,"stamp":"2026-02-22T20:53:33.126774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1539376,"kb_used_data":655248,"kb_used_omap":56,"kb_used_meta":884039,"kb_avail":187204304,"statfs":{"total":193273528320,"available":191697207296,"internally_reserved":0,"allocated":670973952,"data_stored":1039274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58088,"internal_metadata":905256216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001273"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40913,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.252908+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:15.252908+0000","last_peered":"2026-02-22T20:53:15.252908+0000","last_clean":"2026-02-22T20:53:15.252908+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:15.252908+0000","last_undegraded":"2026-02-22T20:53:15.252908+0000","last_fullsized":"2026-02-22T20:53:15.252908+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41270,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253746+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:15.253746+0000","last_peered":"2026-02-22T20:53:15.253746+0000","last_clean":"2026-02-22T20:53:15.253746+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:15.253746+0000","last_undegraded":"2026-02-22T20:53:15.253746+0000","last_fullsized":"2026-02-22T20:53:15.253746+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40514,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253720+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:15.253720+0000","last_peered":"2026-02-22T20:53:15.253720+0000","last_clean":"2026-02-22T20:53:15.253720+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:15.253720+0000","last_undegraded":"2026-02-22T20:53:15.253720+0000","last_fullsized":"2026-02-22T20:53:15.253720+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41833,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253728+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:15.253728+0000","last_peered":"2026-02-22T20:53:15.253728+0000","last_clean":"2026-02-22T20:53:15.253728+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:15.253728+0000","last_undegraded":"2026-02-22T20:53:15.253728+0000","last_fullsized":"2026-02-22T20:53:15.253728+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10135,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.252866+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:15.252866+0000","last_peered":"2026-02-22T20:53:15.252866+0000","last_clean":"2026-02-22T20:53:15.252866+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:15.252866+0000","last_undegraded":"2026-02-22T20:53:15.252866+0000","last_fullsized":"2026-02-22T20:53:15.252866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40774,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253710+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:15.253710+0000","last_peered":"2026-02-22T20:53:15.253710+0000","last_clean":"2026-02-22T20:53:15.253710+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:15.253710+0000","last_undegraded":"2026-02-22T20:53:15.253710+0000","last_fullsized":"2026-02-22T20:53:15.253710+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41065,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253702+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:15.253702+0000","last_peered":"2026-02-22T20:53:15.253702+0000","last_clean":"2026-02-22T20:53:15.253702+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:15.253702+0000","last_undegraded":"2026-02-22T20:53:15.253702+0000","last_fullsized":"2026-02-22T20:53:15.253702+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":41063,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.258082+0000","last_change":"2026-02-22T20:53:15.258045+0000","last_active":"2026-02-22T20:53:15.258082+0000","last_peered":"2026-02-22T20:53:15.258082+0000","last_clean":"2026-02-22T20:53:15.258082+0000","last_became_active":"2026-02-22T20:53:15.254030+0000","last_became_peered":"2026-02-22T20:53:15.254030+0000","last_unstale":"2026-02-22T20:53:15.258082+0000","last_undegraded":"2026-02-22T20:53:15.258082+0000","last_fullsized":"2026-02-22T20:53:15.258082+0000","mapping_epoch":46,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:27:41.800790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1000562688,"data_stored":1115361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051610,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182429,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705914,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769688,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":442019,"kb_avail":93602152,"statfs":{"total":96636764160,"available":95848603648,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":452628108},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705915,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769688,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":442019,"kb_avail":93602152,"statfs":{"total":96636764160,"available":95848603648,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":452628108},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288387072,"data_stored":299648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:33.562 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-22T20:53:33.563 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:33.743 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:33.744 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:33.755 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":658,"stamp":"2026-02-22T20:53:33.126774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1539376,"kb_used_data":655248,"kb_used_omap":56,"kb_used_meta":884039,"kb_avail":187204304,"statfs":{"total":193273528320,"available":191697207296,"internally_reserved":0,"allocated":670973952,"data_stored":1039274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58088,"internal_metadata":905256216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001273"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40913,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.252908+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:15.252908+0000","last_peered":"2026-02-22T20:53:15.252908+0000","last_clean":"2026-02-22T20:53:15.252908+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:15.252908+0000","last_undegraded":"2026-02-22T20:53:15.252908+0000","last_fullsized":"2026-02-22T20:53:15.252908+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41270,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253746+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:15.253746+0000","last_peered":"2026-02-22T20:53:15.253746+0000","last_clean":"2026-02-22T20:53:15.253746+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:15.253746+0000","last_undegraded":"2026-02-22T20:53:15.253746+0000","last_fullsized":"2026-02-22T20:53:15.253746+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40514,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253720+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:15.253720+0000","last_peered":"2026-02-22T20:53:15.253720+0000","last_clean":"2026-02-22T20:53:15.253720+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:15.253720+0000","last_undegraded":"2026-02-22T20:53:15.253720+0000","last_fullsized":"2026-02-22T20:53:15.253720+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41833,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253728+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:15.253728+0000","last_peered":"2026-02-22T20:53:15.253728+0000","last_clean":"2026-02-22T20:53:15.253728+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:15.253728+0000","last_undegraded":"2026-02-22T20:53:15.253728+0000","last_fullsized":"2026-02-22T20:53:15.253728+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10135,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.252866+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:15.252866+0000","last_peered":"2026-02-22T20:53:15.252866+0000","last_clean":"2026-02-22T20:53:15.252866+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:15.252866+0000","last_undegraded":"2026-02-22T20:53:15.252866+0000","last_fullsized":"2026-02-22T20:53:15.252866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40774,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253710+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:15.253710+0000","last_peered":"2026-02-22T20:53:15.253710+0000","last_clean":"2026-02-22T20:53:15.253710+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:15.253710+0000","last_undegraded":"2026-02-22T20:53:15.253710+0000","last_fullsized":"2026-02-22T20:53:15.253710+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41065,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253702+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:15.253702+0000","last_peered":"2026-02-22T20:53:15.253702+0000","last_clean":"2026-02-22T20:53:15.253702+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:15.253702+0000","last_undegraded":"2026-02-22T20:53:15.253702+0000","last_fullsized":"2026-02-22T20:53:15.253702+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":41063,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.258082+0000","last_change":"2026-02-22T20:53:15.258045+0000","last_active":"2026-02-22T20:53:15.258082+0000","last_peered":"2026-02-22T20:53:15.258082+0000","last_clean":"2026-02-22T20:53:15.258082+0000","last_became_active":"2026-02-22T20:53:15.254030+0000","last_became_peered":"2026-02-22T20:53:15.254030+0000","last_unstale":"2026-02-22T20:53:15.258082+0000","last_undegraded":"2026-02-22T20:53:15.258082+0000","last_fullsized":"2026-02-22T20:53:15.258082+0000","mapping_epoch":46,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:27:41.800790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1000562688,"data_stored":1115361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051610,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182429,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705914,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769688,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":442019,"kb_avail":93602152,"statfs":{"total":96636764160,"available":95848603648,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":452628108},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705915,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769688,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":442019,"kb_avail":93602152,"statfs":{"total":96636764160,"available":95848603648,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":452628108},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288387072,"data_stored":299648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:33.756 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T20:53:33.934 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:33.935 INFO:teuthology.orchestra.run.trial084.stdout:{"epoch":47,"fsid":"a4ea559c-fa00-404b-b21c-f34ee9bd0795","created":"2026-02-22T20:33:03.270442+0000","modified":"2026-02-22T20:53:15.248582+0000","last_up_change":"2026-02-22T20:51:24.166635+0000","last_in_change":"2026-02-22T20:50:33.126438+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-22T20:33:12.644417+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":0.5,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ca475199-167f-4aba-960a-1e3782214a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6802","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6803","nonce":1867649424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6804","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6805","nonce":1867649424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6808","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6809","nonce":1867649424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6806","nonce":1867649424},{"type":"v1","addr":"10.20.193.84:6807","nonce":1867649424}]},"public_addr":"10.20.193.84:6803/1867649424","cluster_addr":"10.20.193.84:6805/1867649424","heartbeat_back_addr":"10.20.193.84:6809/1867649424","heartbeat_front_addr":"10.20.193.84:6807/1867649424","state":["exists","up"]},{"osd":1,"uuid":"32df476a-2b69-493e-a636-1ba8bf5268a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6811","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6813","nonce":2769530292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6815","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6818","nonce":2769530292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6824","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6825","nonce":2769530292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6820","nonce":2769530292},{"type":"v1","addr":"10.20.193.84:6822","nonce":2769530292}]},"public_addr":"10.20.193.84:6813/2769530292","cluster_addr":"10.20.193.84:6818/2769530292","heartbeat_back_addr":"10.20.193.84:6825/2769530292","heartbeat_front_addr":"10.20.193.84:6822/2769530292","state":["exists","up"]},{"osd":2,"uuid":"196c3661-b3be-46a3-9f14-7908d9003447","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":45,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6810","nonce":524818824},{"type":"v1","addr":"10.20.193.84:6812","nonce":524818824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6814","nonce":524818824},{"type":"v1","addr":"10.20.193.84:6816","nonce":524818824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6821","nonce":524818824},{"type":"v1","addr":"10.20.193.84:6823","nonce":524818824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6817","nonce":524818824},{"type":"v1","addr":"10.20.193.84:6819","nonce":524818824}]},"public_addr":"10.20.193.84:6812/524818824","cluster_addr":"10.20.193.84:6816/524818824","heartbeat_back_addr":"10.20.193.84:6823/524818824","heartbeat_front_addr":"10.20.193.84:6819/524818824","state":["exists","up"]},{"osd":3,"uuid":"bdc6271f-361f-4362-b050-080c613ab56c","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":29,"up_thru":19,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6826","nonce":1452505971},{"type":"v1","addr":"10.20.193.84:6827","nonce":1452505971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6828","nonce":1452505971},{"type":"v1","addr":"10.20.193.84:6829","nonce":1452505971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6832","nonce":1452505971},{"type":"v1","addr":"10.20.193.84:6833","nonce":1452505971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.84:6830","nonce":1452505971},{"type":"v1","addr":"10.20.193.84:6831","nonce":1452505971}]},"public_addr":"10.20.193.84:6827/1452505971","cluster_addr":"10.20.193.84:6829/1452505971","heartbeat_back_addr":"10.20.193.84:6833/1452505971","heartbeat_front_addr":"10.20.193.84:6831/1452505971","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T20:33:07.139199+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T20:33:07.187223+0000","dead_epoch":0},{"osd":2,"down_stamp":"2026-02-22T20:51:04.235695+0000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":65536,"last_purged_snaps_scrub":"2026-02-22T20:33:07.198419+0000","dead_epoch":0},{"osd":3,"down_stamp":"2026-02-22T20:51:04.235695+0000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":65536,"last_purged_snaps_scrub":"2026-02-22T20:33:07.149639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T20:53:34.947 INFO:tasks.ceph:Scrubbing osd.0 2026-02-22T20:53:34.947 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-22T20:53:35.046 INFO:teuthology.orchestra.run.trial084.stdout:{ 2026-02-22T20:53:35.046 INFO:teuthology.orchestra.run.trial084.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T20:53:35.046 INFO:teuthology.orchestra.run.trial084.stdout:} 2026-02-22T20:53:35.057 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-22T20:53:35.237 INFO:teuthology.orchestra.run.trial084.stderr:instructed osd(s) 0 to deep-scrub 2026-02-22T20:53:35.250 INFO:tasks.ceph:Scrubbing osd.1 2026-02-22T20:53:35.250 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-22T20:53:35.347 INFO:teuthology.orchestra.run.trial084.stdout:{ 2026-02-22T20:53:35.347 INFO:teuthology.orchestra.run.trial084.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T20:53:35.347 INFO:teuthology.orchestra.run.trial084.stdout:} 2026-02-22T20:53:35.357 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-22T20:53:35.536 INFO:teuthology.orchestra.run.trial084.stderr:instructed osd(s) 1 to deep-scrub 2026-02-22T20:53:35.549 INFO:tasks.ceph:Scrubbing osd.2 2026-02-22T20:53:35.549 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-22T20:53:35.645 INFO:teuthology.orchestra.run.trial084.stdout:{ 2026-02-22T20:53:35.645 INFO:teuthology.orchestra.run.trial084.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T20:53:35.646 INFO:teuthology.orchestra.run.trial084.stdout:} 2026-02-22T20:53:35.658 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-22T20:53:35.841 INFO:teuthology.orchestra.run.trial084.stderr:instructed osd(s) 2 to deep-scrub 2026-02-22T20:53:35.854 INFO:tasks.ceph:Scrubbing osd.3 2026-02-22T20:53:35.855 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-22T20:53:35.949 INFO:teuthology.orchestra.run.trial084.stdout:{ 2026-02-22T20:53:35.950 INFO:teuthology.orchestra.run.trial084.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T20:53:35.950 INFO:teuthology.orchestra.run.trial084.stdout:} 2026-02-22T20:53:35.960 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-22T20:53:36.144 INFO:teuthology.orchestra.run.trial084.stderr:instructed osd(s) 3 to deep-scrub 2026-02-22T20:53:36.161 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:36.357 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:36.357 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:36.372 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":659,"stamp":"2026-02-22T20:53:35.126979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1539376,"kb_used_data":655248,"kb_used_omap":56,"kb_used_meta":884039,"kb_avail":187204304,"statfs":{"total":193273528320,"available":191697207296,"internally_reserved":0,"allocated":670973952,"data_stored":1039274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58088,"internal_metadata":905256216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001261"},"pg_stats":[{"pgid":"1.7","version":"17'10205","reported_seq":40913,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.252908+0000","last_change":"2026-02-22T20:53:00.227588+0000","last_active":"2026-02-22T20:53:15.252908+0000","last_peered":"2026-02-22T20:53:15.252908+0000","last_clean":"2026-02-22T20:53:15.252908+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:15.252908+0000","last_undegraded":"2026-02-22T20:53:15.252908+0000","last_fullsized":"2026-02-22T20:53:15.252908+0000","mapping_epoch":42,"log_start":"17'205","ondisk_log_start":"17'205","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:54:53.960682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"17'10316","reported_seq":41270,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253746+0000","last_change":"2026-02-22T20:52:51.208395+0000","last_active":"2026-02-22T20:53:15.253746+0000","last_peered":"2026-02-22T20:53:15.253746+0000","last_clean":"2026-02-22T20:53:15.253746+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:15.253746+0000","last_undegraded":"2026-02-22T20:53:15.253746+0000","last_fullsized":"2026-02-22T20:53:15.253746+0000","mapping_epoch":40,"log_start":"17'316","ondisk_log_start":"17'316","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T20:47:29.727936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"17'10101","reported_seq":40514,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253720+0000","last_change":"2026-02-22T20:52:20.159926+0000","last_active":"2026-02-22T20:53:15.253720+0000","last_peered":"2026-02-22T20:53:15.253720+0000","last_clean":"2026-02-22T20:53:15.253720+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:15.253720+0000","last_undegraded":"2026-02-22T20:53:15.253720+0000","last_fullsized":"2026-02-22T20:53:15.253720+0000","mapping_epoch":34,"log_start":"17'101","ondisk_log_start":"17'101","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:45:39.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"17'10432","reported_seq":41833,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253728+0000","last_change":"2026-02-22T20:52:42.195666+0000","last_active":"2026-02-22T20:53:15.253728+0000","last_peered":"2026-02-22T20:53:15.253728+0000","last_clean":"2026-02-22T20:53:15.253728+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:15.253728+0000","last_undegraded":"2026-02-22T20:53:15.253728+0000","last_fullsized":"2026-02-22T20:53:15.253728+0000","mapping_epoch":38,"log_start":"17'432","ondisk_log_start":"17'432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:54:09.870946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"17'10055","reported_seq":10135,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.252866+0000","last_change":"2026-02-22T20:52:09.142068+0000","last_active":"2026-02-22T20:53:15.252866+0000","last_peered":"2026-02-22T20:53:15.252866+0000","last_clean":"2026-02-22T20:53:15.252866+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:15.252866+0000","last_undegraded":"2026-02-22T20:53:15.252866+0000","last_fullsized":"2026-02-22T20:53:15.252866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:50:36.801653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'10169","reported_seq":40774,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253710+0000","last_change":"2026-02-22T20:52:31.180357+0000","last_active":"2026-02-22T20:53:15.253710+0000","last_peered":"2026-02-22T20:53:15.253710+0000","last_clean":"2026-02-22T20:53:15.253710+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:15.253710+0000","last_undegraded":"2026-02-22T20:53:15.253710+0000","last_fullsized":"2026-02-22T20:53:15.253710+0000","mapping_epoch":36,"log_start":"17'169","ondisk_log_start":"17'169","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:43:36.690347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"17'10238","reported_seq":41065,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.253702+0000","last_change":"2026-02-22T20:53:08.241073+0000","last_active":"2026-02-22T20:53:15.253702+0000","last_peered":"2026-02-22T20:53:15.253702+0000","last_clean":"2026-02-22T20:53:15.253702+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:15.253702+0000","last_undegraded":"2026-02-22T20:53:15.253702+0000","last_fullsized":"2026-02-22T20:53:15.253702+0000","mapping_epoch":44,"log_start":"17'238","ondisk_log_start":"17'238","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:36:01.583519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'10239","reported_seq":41063,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:15.258082+0000","last_change":"2026-02-22T20:53:15.258045+0000","last_active":"2026-02-22T20:53:15.258082+0000","last_peered":"2026-02-22T20:53:15.258082+0000","last_clean":"2026-02-22T20:53:15.258082+0000","last_became_active":"2026-02-22T20:53:15.254030+0000","last_became_peered":"2026-02-22T20:53:15.254030+0000","last_unstale":"2026-02-22T20:53:15.258082+0000","last_undegraded":"2026-02-22T20:53:15.258082+0000","last_fullsized":"2026-02-22T20:53:15.258082+0000","mapping_epoch":46,"log_start":"17'239","ondisk_log_start":"17'239","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":0,"log_size":10000,"log_dups_size":0,"ondisk_log_size":10000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:27:41.800790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1000562688,"data_stored":1115361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80055,"ondisk_log_size":80055,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051611,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182429,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705915,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769688,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":442019,"kb_avail":93602152,"statfs":{"total":96636764160,"available":95848603648,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":452628108},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705916,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769688,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":442019,"kb_avail":93602152,"statfs":{"total":96636764160,"available":95848603648,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":452628108},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288387072,"data_stored":299648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:36.379 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:36.379 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:36.379 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:36.379 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:36.379 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:36.380 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:36.380 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:36.380 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:36.380 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-22T20:53:56.381 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:53:56.584 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:53:56.585 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:53:56.601 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":669,"stamp":"2026-02-22T20:53:55.129664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80487,"ondisk_log_size":80487,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1792048,"kb_used_data":655248,"kb_used_omap":56,"kb_used_meta":1136711,"kb_avail":186951632,"statfs":{"total":193273528320,"available":191438471168,"internally_reserved":0,"allocated":670973952,"data_stored":1039274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58088,"internal_metadata":1163992344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,5,8,21,14],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001738"},"pg_stats":[{"pgid":"1.7","version":"47'17074","reported_seq":48244,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-22T20:53:50.147044+0000","last_change":"2026-02-22T20:53:38.107013+0000","last_active":"2026-02-22T20:53:50.147044+0000","last_peered":"2026-02-22T20:53:50.147044+0000","last_clean":"2026-02-22T20:53:50.147044+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:50.147044+0000","last_undegraded":"2026-02-22T20:53:50.147044+0000","last_fullsized":"2026-02-22T20:53:50.147044+0000","mapping_epoch":42,"log_start":"17'7005","ondisk_log_start":"17'7005","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":6869,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 13s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"47'16270","reported_seq":47626,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-22T20:53:50.241994+0000","last_change":"2026-02-22T20:53:40.092368+0000","last_active":"2026-02-22T20:53:50.241994+0000","last_peered":"2026-02-22T20:53:50.241994+0000","last_clean":"2026-02-22T20:53:50.241994+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:50.241994+0000","last_undegraded":"2026-02-22T20:53:50.241994+0000","last_fullsized":"2026-02-22T20:53:50.241994+0000","mapping_epoch":40,"log_start":"17'6216","ondisk_log_start":"17'6216","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":5969,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 11s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"47'16385","reported_seq":47222,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-22T20:53:50.235148+0000","last_change":"2026-02-22T20:53:39.063252+0000","last_active":"2026-02-22T20:53:50.235148+0000","last_peered":"2026-02-22T20:53:50.235148+0000","last_clean":"2026-02-22T20:53:50.235148+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:50.235148+0000","last_undegraded":"2026-02-22T20:53:50.235148+0000","last_fullsized":"2026-02-22T20:53:50.235148+0000","mapping_epoch":34,"log_start":"17'6301","ondisk_log_start":"17'6301","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":6299,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 12s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"47'17436","reported_seq":49294,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-22T20:53:50.231560+0000","last_change":"2026-02-22T20:53:38.027471+0000","last_active":"2026-02-22T20:53:50.231560+0000","last_peered":"2026-02-22T20:53:50.231560+0000","last_clean":"2026-02-22T20:53:50.231560+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:50.231560+0000","last_undegraded":"2026-02-22T20:53:50.231560+0000","last_fullsized":"2026-02-22T20:53:50.231560+0000","mapping_epoch":38,"log_start":"17'7432","ondisk_log_start":"17'7432","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":7004,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 13s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"47'17389","reported_seq":17960,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-22T20:53:50.150124+0000","last_change":"2026-02-22T20:53:37.099505+0000","last_active":"2026-02-22T20:53:50.150124+0000","last_peered":"2026-02-22T20:53:50.150124+0000","last_clean":"2026-02-22T20:53:50.150124+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:50.150124+0000","last_undegraded":"2026-02-22T20:53:50.150124+0000","last_fullsized":"2026-02-22T20:53:50.150124+0000","mapping_epoch":32,"log_start":"17'7300","ondisk_log_start":"17'7300","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":7334,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 14s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"47'17578","reported_seq":48677,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-22T20:53:50.245735+0000","last_change":"2026-02-22T20:53:36.996401+0000","last_active":"2026-02-22T20:53:50.245735+0000","last_peered":"2026-02-22T20:53:50.245735+0000","last_clean":"2026-02-22T20:53:50.245735+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:50.245735+0000","last_undegraded":"2026-02-22T20:53:50.245735+0000","last_fullsized":"2026-02-22T20:53:50.245735+0000","mapping_epoch":36,"log_start":"17'7569","ondisk_log_start":"17'7569","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":7409,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 14s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"47'18622","reported_seq":50013,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-22T20:53:50.222234+0000","last_change":"2026-02-22T20:53:36.022616+0000","last_active":"2026-02-22T20:53:50.222234+0000","last_peered":"2026-02-22T20:53:50.222234+0000","last_clean":"2026-02-22T20:53:50.222234+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:50.222234+0000","last_undegraded":"2026-02-22T20:53:50.222234+0000","last_fullsized":"2026-02-22T20:53:50.222234+0000","mapping_epoch":44,"log_start":"17'8538","ondisk_log_start":"17'8538","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":8399,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 15s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"47'18233","reported_seq":49594,"reported_epoch":47,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-22T20:53:50.146360+0000","last_change":"2026-02-22T20:53:36.136499+0000","last_active":"2026-02-22T20:53:50.146360+0000","last_peered":"2026-02-22T20:53:50.146360+0000","last_clean":"2026-02-22T20:53:50.146360+0000","last_became_active":"2026-02-22T20:53:15.254030+0000","last_became_peered":"2026-02-22T20:53:15.254030+0000","last_unstale":"2026-02-22T20:53:50.146360+0000","last_undegraded":"2026-02-22T20:53:50.146360+0000","last_fullsized":"2026-02-22T20:53:50.146360+0000","mapping_epoch":46,"log_start":"17'8139","ondisk_log_start":"17'8139","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T20:33:12.997318+0000","last_clean_scrub_stamp":"2026-02-22T20:33:12.997318+0000","objects_scrubbed":7994,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 15s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":980901888,"data_stored":1110561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80487,"ondisk_log_size":80487,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051615,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182433,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705918,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":896024,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":568355,"kb_avail":93475816,"statfs":{"total":96636764160,"available":95719235584,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":581996172},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,5,11,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705919,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":896024,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":568355,"kb_avail":93475816,"statfs":{"total":96636764160,"available":95719235584,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":581996172},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,10,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":268726272,"data_stored":294848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:53:56.602 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:56.603 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:56.603 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:56.603 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:56.603 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:56.603 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:56.604 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:56.604 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-22T20:33:12.997318+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=33, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=20, tm_min=53, tm_sec=33, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T20:53:56.604 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-22T20:54:16.605 DEBUG:teuthology.orchestra.run.trial084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T20:54:16.788 INFO:teuthology.orchestra.run.trial084.stdout: 2026-02-22T20:54:16.788 INFO:teuthology.orchestra.run.trial084.stderr:dumped all 2026-02-22T20:54:16.799 INFO:teuthology.orchestra.run.trial084.stdout:{"pg_ready":true,"pg_map":{"version":679,"stamp":"2026-02-22T20:54:15.131641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80209,"ondisk_log_size":80209,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":1864240,"kb_used_data":655248,"kb_used_omap":56,"kb_used_meta":1208903,"kb_avail":186879440,"statfs":{"total":193273528320,"available":191364546560,"internally_reserved":0,"allocated":670973952,"data_stored":1039274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58088,"internal_metadata":1237916952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001109"},"pg_stats":[{"pgid":"1.7","version":"47'20410","reported_seq":51808,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:56.440243+0000","last_change":"2026-02-22T20:53:56.440243+0000","last_active":"2026-02-22T20:53:56.440243+0000","last_peered":"2026-02-22T20:53:56.440243+0000","last_clean":"2026-02-22T20:53:56.440243+0000","last_became_active":"2026-02-22T20:53:00.222728+0000","last_became_peered":"2026-02-22T20:53:00.222728+0000","last_unstale":"2026-02-22T20:53:56.440243+0000","last_undegraded":"2026-02-22T20:53:56.440243+0000","last_fullsized":"2026-02-22T20:53:56.440243+0000","mapping_epoch":42,"log_start":"47'10405","ondisk_log_start":"47'10405","created":10,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20410","last_scrub_stamp":"2026-02-22T20:53:56.440188+0000","last_deep_scrub":"47'20410","last_deep_scrub_stamp":"2026-02-22T20:53:56.440188+0000","last_clean_scrub_stamp":"2026-02-22T20:53:56.440188+0000","objects_scrubbed":10205,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:41:30.800091+0000","scrub_duration":18333,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10205,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":10205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10205,"num_bytes_recovered":10205,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"47'20632","reported_seq":52283,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:57.459983+0000","last_change":"2026-02-22T20:53:57.459983+0000","last_active":"2026-02-22T20:53:57.459983+0000","last_peered":"2026-02-22T20:53:57.459983+0000","last_clean":"2026-02-22T20:53:57.459983+0000","last_became_active":"2026-02-22T20:52:51.203875+0000","last_became_peered":"2026-02-22T20:52:51.203875+0000","last_unstale":"2026-02-22T20:53:57.459983+0000","last_undegraded":"2026-02-22T20:53:57.459983+0000","last_fullsized":"2026-02-22T20:53:57.459983+0000","mapping_epoch":40,"log_start":"47'10616","ondisk_log_start":"47'10616","created":10,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20632","last_scrub_stamp":"2026-02-22T20:53:57.459951+0000","last_deep_scrub":"47'20632","last_deep_scrub_stamp":"2026-02-22T20:53:57.459951+0000","last_clean_scrub_stamp":"2026-02-22T20:53:57.459951+0000","objects_scrubbed":10316,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T21:19:35.335919+0000","scrub_duration":17368,"objects_trimmed":0,"snaptrim_duration":0.00077982699999999997,"stat_sum":{"num_bytes":71448,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10282,"num_bytes_recovered":71414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"47'20202","reported_seq":51298,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:57.035745+0000","last_change":"2026-02-22T20:53:57.035745+0000","last_active":"2026-02-22T20:53:57.035745+0000","last_peered":"2026-02-22T20:53:57.035745+0000","last_clean":"2026-02-22T20:53:57.035745+0000","last_became_active":"2026-02-22T20:52:20.155258+0000","last_became_peered":"2026-02-22T20:52:20.155258+0000","last_unstale":"2026-02-22T20:53:57.035745+0000","last_undegraded":"2026-02-22T20:53:57.035745+0000","last_fullsized":"2026-02-22T20:53:57.035745+0000","mapping_epoch":34,"log_start":"47'10201","ondisk_log_start":"47'10201","created":10,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20202","last_scrub_stamp":"2026-02-22T20:53:57.035710+0000","last_deep_scrub":"47'20202","last_deep_scrub_stamp":"2026-02-22T20:53:57.035710+0000","last_clean_scrub_stamp":"2026-02-22T20:53:57.035710+0000","objects_scrubbed":10101,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:59:14.818156+0000","scrub_duration":17973,"objects_trimmed":0,"snaptrim_duration":0.000206202,"stat_sum":{"num_bytes":40690,"num_objects":10101,"num_object_clones":0,"num_object_copies":20202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10101,"num_bytes_recovered":40690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"47'20864","reported_seq":52970,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:56.653540+0000","last_change":"2026-02-22T20:53:56.653540+0000","last_active":"2026-02-22T20:53:56.653540+0000","last_peered":"2026-02-22T20:53:56.653540+0000","last_clean":"2026-02-22T20:53:56.653540+0000","last_became_active":"2026-02-22T20:52:42.191499+0000","last_became_peered":"2026-02-22T20:52:42.191499+0000","last_unstale":"2026-02-22T20:53:56.653540+0000","last_undegraded":"2026-02-22T20:53:56.653540+0000","last_fullsized":"2026-02-22T20:53:56.653540+0000","mapping_epoch":38,"log_start":"47'10832","ondisk_log_start":"47'10832","created":10,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20864","last_scrub_stamp":"2026-02-22T20:53:56.653461+0000","last_deep_scrub":"47'20864","last_deep_scrub_stamp":"2026-02-22T20:53:56.653461+0000","last_clean_scrub_stamp":"2026-02-22T20:53:56.653461+0000","objects_scrubbed":10432,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:47:40.700063+0000","scrub_duration":18626,"objects_trimmed":0,"snaptrim_duration":0.00041580099999999999,"stat_sum":{"num_bytes":71564,"num_objects":10432,"num_object_clones":0,"num_object_copies":20864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10432,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10432,"num_bytes_recovered":71564,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"47'20110","reported_seq":20870,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:55.640947+0000","last_change":"2026-02-22T20:53:55.640947+0000","last_active":"2026-02-22T20:53:55.640947+0000","last_peered":"2026-02-22T20:53:55.640947+0000","last_clean":"2026-02-22T20:53:55.640947+0000","last_became_active":"2026-02-22T20:52:09.136781+0000","last_became_peered":"2026-02-22T20:52:09.136781+0000","last_unstale":"2026-02-22T20:53:55.640947+0000","last_undegraded":"2026-02-22T20:53:55.640947+0000","last_fullsized":"2026-02-22T20:53:55.640947+0000","mapping_epoch":32,"log_start":"47'10100","ondisk_log_start":"47'10100","created":10,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20110","last_scrub_stamp":"2026-02-22T20:53:55.640883+0000","last_deep_scrub":"47'20110","last_deep_scrub_stamp":"2026-02-22T20:53:55.640883+0000","last_clean_scrub_stamp":"2026-02-22T20:53:55.640883+0000","objects_scrubbed":10055,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:33:49.331872+0000","scrub_duration":18542,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8397,"num_bytes_recovered":38962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"47'20337","reported_seq":51629,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:55.686762+0000","last_change":"2026-02-22T20:53:55.686762+0000","last_active":"2026-02-22T20:53:55.686762+0000","last_peered":"2026-02-22T20:53:55.686762+0000","last_clean":"2026-02-22T20:53:55.686762+0000","last_became_active":"2026-02-22T20:52:31.176138+0000","last_became_peered":"2026-02-22T20:52:31.176138+0000","last_unstale":"2026-02-22T20:53:55.686762+0000","last_undegraded":"2026-02-22T20:53:55.686762+0000","last_fullsized":"2026-02-22T20:53:55.686762+0000","mapping_epoch":36,"log_start":"47'10269","ondisk_log_start":"47'10269","created":10,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20337","last_scrub_stamp":"2026-02-22T20:53:55.686675+0000","last_deep_scrub":"47'20337","last_deep_scrub_stamp":"2026-02-22T20:53:55.686675+0000","last_clean_scrub_stamp":"2026-02-22T20:53:55.686675+0000","objects_scrubbed":10168,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:37:58.856841+0000","scrub_duration":18691,"objects_trimmed":0,"snaptrim_duration":0.00057745099999999998,"stat_sum":{"num_bytes":40752,"num_objects":10168,"num_object_clones":0,"num_object_copies":20336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":10197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10169,"num_bytes_recovered":40771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"47'20476","reported_seq":51994,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:54.095298+0000","last_change":"2026-02-22T20:53:54.095298+0000","last_active":"2026-02-22T20:53:54.095298+0000","last_peered":"2026-02-22T20:53:54.095298+0000","last_clean":"2026-02-22T20:53:54.095298+0000","last_became_active":"2026-02-22T20:53:08.237151+0000","last_became_peered":"2026-02-22T20:53:08.237151+0000","last_unstale":"2026-02-22T20:53:54.095298+0000","last_undegraded":"2026-02-22T20:53:54.095298+0000","last_fullsized":"2026-02-22T20:53:54.095298+0000","mapping_epoch":44,"log_start":"47'10438","ondisk_log_start":"47'10438","created":10,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20476","last_scrub_stamp":"2026-02-22T20:53:54.095199+0000","last_deep_scrub":"47'20476","last_deep_scrub_stamp":"2026-02-22T20:53:54.095199+0000","last_clean_scrub_stamp":"2026-02-22T20:53:54.095199+0000","objects_scrubbed":10238,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:37:14.448792+0000","scrub_duration":18072,"objects_trimmed":0,"snaptrim_duration":0.00065876200000000002,"stat_sum":{"num_bytes":25521,"num_objects":10238,"num_object_clones":0,"num_object_copies":20476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10238,"num_write_kb":10252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10238,"num_bytes_recovered":25521,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"47'20478","reported_seq":51993,"reported_epoch":47,"state":"active+clean","last_fresh":"2026-02-22T20:53:54.762952+0000","last_change":"2026-02-22T20:53:54.762952+0000","last_active":"2026-02-22T20:53:54.762952+0000","last_peered":"2026-02-22T20:53:54.762952+0000","last_clean":"2026-02-22T20:53:54.762952+0000","last_became_active":"2026-02-22T20:53:15.254030+0000","last_became_peered":"2026-02-22T20:53:15.254030+0000","last_unstale":"2026-02-22T20:53:54.762952+0000","last_undegraded":"2026-02-22T20:53:54.762952+0000","last_fullsized":"2026-02-22T20:53:54.762952+0000","mapping_epoch":46,"log_start":"47'10439","ondisk_log_start":"47'10439","created":10,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20478","last_scrub_stamp":"2026-02-22T20:53:54.762886+0000","last_deep_scrub":"47'20478","last_deep_scrub_stamp":"2026-02-22T20:53:54.762886+0000","last_clean_scrub_stamp":"2026-02-22T20:53:54.762886+0000","objects_scrubbed":10239,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:38:10.828399+0000","scrub_duration":18627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71371,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10239,"num_bytes_recovered":71371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":372171,"num_objects":81754,"num_object_clones":0,"num_object_copies":163508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81754,"num_write_kb":82020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80063,"num_bytes_recovered":370498,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":961228800,"data_stored":1090478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80209,"ondisk_log_size":80209,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":0,"seq":124554051619,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":107374182437,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705922,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":932120,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":604451,"kb_avail":93439720,"statfs":{"total":96636764160,"available":95682273280,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":618958476},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705923,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":932120,"kb_used_data":327624,"kb_used_omap":28,"kb_used_meta":604451,"kb_avail":93439720,"statfs":{"total":96636764160,"available":95682273280,"internally_reserved":0,"allocated":335486976,"data_stored":519637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":618958476},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335093760,"data_stored":372171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":249053184,"data_stored":274765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41988096,"data_stored":71371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T20:54:16.801 DEBUG:teuthology.orchestra.run.trial084:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-22T20:54:16.995 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-22T20:54:16.995 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-22T20:54:16.995 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-22T20:54:16.995 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T20:54:17.083 INFO:tasks.ceph.osd.0:Stopped 2026-02-22T20:54:17.083 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-22T20:54:17.083 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T20:54:17.168 INFO:tasks.ceph.osd.1:Stopped 2026-02-22T20:54:17.168 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-22T20:54:17.168 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T20:54:17.249 INFO:tasks.ceph.osd.2:Stopped 2026-02-22T20:54:17.249 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-22T20:54:17.249 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T20:54:17.302 INFO:tasks.ceph.osd.3:Stopped 2026-02-22T20:54:17.302 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-22T20:54:17.302 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-22T20:54:17.302 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T20:54:17.322 INFO:tasks.ceph.mgr.x:Stopped 2026-02-22T20:54:17.322 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-22T20:54:17.323 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-22T20:54:17.323 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T20:54:17.372 INFO:tasks.ceph.mon.a:Stopped 2026-02-22T20:54:17.372 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-22T20:54:17.373 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T20:54:17.421 INFO:tasks.ceph.mon.b:Stopped 2026-02-22T20:54:17.421 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-22T20:54:17.421 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T20:54:17.473 INFO:tasks.ceph.mon.c:Stopped 2026-02-22T20:54:17.473 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-22T20:54:17.473 DEBUG:teuthology.orchestra.run.trial084:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v 'but it is still running' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSDMAP_FLAGS\)' | grep -E -v '\(OSD_' | grep -E -v '\(PG_' | grep -E -v '\(OBJECT_' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2026-02-22T20:54:17.541 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial084.front.sepia.ceph.com 2026-02-22T20:54:17.541 DEBUG:teuthology.orchestra.run.trial084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-22T20:54:21.000 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial084.front.sepia.ceph.com 2026-02-22T20:54:21.000 DEBUG:teuthology.orchestra.run.trial084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-22T20:54:21.070 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial084.front.sepia.ceph.com 2026-02-22T20:54:21.071 DEBUG:teuthology.orchestra.run.trial084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-22T20:54:21.118 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial084.front.sepia.ceph.com 2026-02-22T20:54:21.118 DEBUG:teuthology.orchestra.run.trial084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-22T20:54:21.166 INFO:tasks.ceph:Archiving mon data... 2026-02-22T20:54:21.169 DEBUG:teuthology.misc:Transferring archived files from trial084:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/nmordech-2026-02-22_19:44:01-rados-wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle-distro-default-trial/64522/data/mon.a.tgz 2026-02-22T20:54:21.169 DEBUG:teuthology.orchestra.run.trial084:> mktemp 2026-02-22T20:54:21.188 INFO:teuthology.orchestra.run.trial084.stdout:/tmp/tmp.fSfzdQrWPk 2026-02-22T20:54:21.188 DEBUG:teuthology.orchestra.run.trial084:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.fSfzdQrWPk 2026-02-22T20:54:21.286 DEBUG:teuthology.orchestra.run.trial084:> sudo chmod 0666 /tmp/tmp.fSfzdQrWPk 2026-02-22T20:54:21.373 DEBUG:teuthology.orchestra.remote:trial084:/tmp/tmp.fSfzdQrWPk is 283KB 2026-02-22T20:54:21.462 DEBUG:teuthology.orchestra.run.trial084:> rm -fr /tmp/tmp.fSfzdQrWPk 2026-02-22T20:54:21.481 DEBUG:teuthology.misc:Transferring archived files from trial084:/var/lib/ceph/mon/ceph-b to /home/teuthworker/mnt/teuthology/nmordech-2026-02-22_19:44:01-rados-wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle-distro-default-trial/64522/data/mon.b.tgz 2026-02-22T20:54:21.481 DEBUG:teuthology.orchestra.run.trial084:> mktemp 2026-02-22T20:54:21.538 INFO:teuthology.orchestra.run.trial084.stdout:/tmp/tmp.pTzriHFFs6 2026-02-22T20:54:21.539 DEBUG:teuthology.orchestra.run.trial084:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.pTzriHFFs6 2026-02-22T20:54:21.639 DEBUG:teuthology.orchestra.run.trial084:> sudo chmod 0666 /tmp/tmp.pTzriHFFs6 2026-02-22T20:54:21.724 DEBUG:teuthology.orchestra.remote:trial084:/tmp/tmp.pTzriHFFs6 is 299KB 2026-02-22T20:54:21.818 DEBUG:teuthology.orchestra.run.trial084:> rm -fr /tmp/tmp.pTzriHFFs6 2026-02-22T20:54:21.837 DEBUG:teuthology.misc:Transferring archived files from trial084:/var/lib/ceph/mon/ceph-c to /home/teuthworker/mnt/teuthology/nmordech-2026-02-22_19:44:01-rados-wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle-distro-default-trial/64522/data/mon.c.tgz 2026-02-22T20:54:21.837 DEBUG:teuthology.orchestra.run.trial084:> mktemp 2026-02-22T20:54:21.893 INFO:teuthology.orchestra.run.trial084.stdout:/tmp/tmp.FXzbi4SmOq 2026-02-22T20:54:21.893 DEBUG:teuthology.orchestra.run.trial084:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.FXzbi4SmOq 2026-02-22T20:54:21.992 DEBUG:teuthology.orchestra.run.trial084:> sudo chmod 0666 /tmp/tmp.FXzbi4SmOq 2026-02-22T20:54:22.077 DEBUG:teuthology.orchestra.remote:trial084:/tmp/tmp.FXzbi4SmOq is 299KB 2026-02-22T20:54:22.164 DEBUG:teuthology.orchestra.run.trial084:> rm -fr /tmp/tmp.FXzbi4SmOq 2026-02-22T20:54:22.181 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-22T20:54:22.181 DEBUG:teuthology.orchestra.run.trial084:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-22T20:54:22.307 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-22T20:54:22.307 INFO:tasks.ceph:Archiving crash dumps... 2026-02-22T20:54:22.311 DEBUG:teuthology.misc:Transferring archived files from trial084:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/nmordech-2026-02-22_19:44:01-rados-wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle-distro-default-trial/64522/remote/trial084/crash 2026-02-22T20:54:22.312 DEBUG:teuthology.orchestra.run.trial084:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-22T20:54:22.374 INFO:tasks.ceph:Compressing logs... 2026-02-22T20:54:22.374 DEBUG:teuthology.orchestra.run.trial084:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T20:54:22.400 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67443.log 2026-02-22T20:54:22.402 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93056.log 2026-02-22T20:54:22.402 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105259.log 2026-02-22T20:54:22.403 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67443.log.gz 2026-02-22T20:54:22.403 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60481.log 2026-02-22T20:54:22.403 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93056.log.gz 2026-02-22T20:54:22.403 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98599.log 2026-02-22T20:54:22.403 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105259.log.gz 2026-02-22T20:54:22.403 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62783.log 2026-02-22T20:54:22.403 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60481.log.gz 2026-02-22T20:54:22.403 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84354.log 2026-02-22T20:54:22.403 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98599.log.gz 2026-02-22T20:54:22.403 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62783.log.gz 2026-02-22T20:54:22.403 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72565.log 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95409.log 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84354.log.gz 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72565.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72565.log.gz -5 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.100567.log 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95409.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95409.log.gz -5 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.65291.log 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100749.log 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100567.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100567.log.gz 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65291.log.gz 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84816.log 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103366.log 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100749.log.gz 2026-02-22T20:54:22.407 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84816.log.gz 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45216.log 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103366.log.gz 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46339.log 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62272.log 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45216.log.gz 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46339.log.gz 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92646.log 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62272.log.gz 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79494.log 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104338.log 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92646.log.gz 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55822.log/var/log/ceph/ceph-client.admin.79494.log: 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79494.log.gz 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104338.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104338.log.gz 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104798.log 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92033.log 2026-02-22T20:54:22.408 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55822.log.gz 2026-02-22T20:54:22.409 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110689.log 2026-02-22T20:54:22.409 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104798.log.gz 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91519.log 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92033.log.gz 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110689.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.110689.log.gz /var/log/ceph/ceph-client.admin.75893.log 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr: 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80772.log 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91519.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91519.log.gz 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75893.log.gz 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53556.log 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104747.log 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80772.log.gz 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53556.log.gz 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65855.log 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91162.log 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104747.log.gz 2026-02-22T20:54:22.413 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67853.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65855.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65855.log.gz 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55667.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91162.log.gz 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56743.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67853.log.gz 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52533.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55667.log.gz 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56743.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56743.log.gz 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105515.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52533.log.gz 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65035.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86913.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105515.log.gz 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65035.log.gz 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97610.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86913.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86913.log.gz 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59407.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60225.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97610.log.gz 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61811.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59407.log.gz 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60225.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60225.log.gz 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74715.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89422.log 2026-02-22T20:54:22.414 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61811.log.gz 2026-02-22T20:54:22.415 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85992.log 2026-02-22T20:54:22.415 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74715.log.gz 2026-02-22T20:54:22.415 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89422.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89422.log.gz 2026-02-22T20:54:22.415 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74817.log 2026-02-22T20:54:22.415 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58434.log 2026-02-22T20:54:22.415 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85992.log.gz 2026-02-22T20:54:22.415 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83691.log 2026-02-22T20:54:22.415 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74817.log.gz 2026-02-22T20:54:22.417 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58434.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.58434.log.gz -- 2026-02-22T20:54:22.417 INFO:teuthology.orchestra.run.trial084.stderr: /var/log/ceph/ceph-client.admin.62221.log 2026-02-22T20:54:22.417 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94642.log 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83691.log.gz 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62221.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62221.log.gzgzip 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.83382.log 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108741.log 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94642.log.gz 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83382.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83382.log.gz 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56333.log 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107000.log 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108741.log.gz 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93311.log 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56333.log.gz 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95358.log 2026-02-22T20:54:22.418 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107000.log.gz 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93311.log.gz 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73640.log 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76931.log 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95358.log.gz 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42574.log 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73640.log.gz 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57254.log 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76931.log.gz 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42574.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42574.log.gz -5 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.86145.log 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67392.log 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57254.log.gz 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86145.log.gz 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66779.log 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97047.log 2026-02-22T20:54:22.419 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67392.log.gz 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77652.log 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66779.log.gz 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97047.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.97047.log.gz -5 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.74358.log 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57049.log 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77652.log.gz 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75279.log 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74358.log.gz 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57049.log.gz 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70873.log 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61249.log 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75279.log.gz 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77288.log 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70873.log.gz 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61249.log.gz 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99167.log 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59458.log 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77288.log.gz 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99305.log 2026-02-22T20:54:22.422 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99167.log.gz 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71232.log 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59458.log.gz 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99305.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99305.log.gz 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42436.log 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71232.log.gz 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83331.log 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59509.log 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42436.log.gz 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83331.log.gz 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62681.log 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108279.log 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59509.log.gz 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62170.log 2026-02-22T20:54:22.423 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62681.log.gz 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103621.log 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108279.log.gz 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62015.log 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62170.log.gz 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95153.log 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103621.log.gz 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90496.log 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62015.log.gz 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89113.log 0.0% 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95153.log.gz 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90496.log.gz 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86658.log 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86760.log 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89113.log.gz 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86658.log.gz 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46083.log 2026-02-22T20:54:22.425 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86760.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.86760.log.gz --verbose 2026-02-22T20:54:22.426 INFO:teuthology.orchestra.run.trial084.stderr: -- /var/log/ceph/ceph-client.admin.51306.log 2026-02-22T20:54:22.426 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83280.log 2026-02-22T20:54:22.426 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46083.log.gz 2026-02-22T20:54:22.426 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72462.log 2026-02-22T20:54:22.426 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51306.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51306.log.gz 2026-02-22T20:54:22.426 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83280.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83280.log.gz 2026-02-22T20:54:22.426 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44757.log 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109559.log 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72462.log.gz 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44757.log.gz 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108023.log 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100516.log 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109559.log.gz 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108023.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108023.log.gz 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54387.log 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100618.log 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100516.log.gz 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54387.log.gz 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93260.log 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100618.log.gz 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87168.log 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97508.log 2026-02-22T20:54:22.427 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93260.log.gz 2026-02-22T20:54:22.430 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5/var/log/ceph/ceph-client.admin.87168.log: --verbose -- /var/log/ceph/ceph-client.admin.64216.log 2026-02-22T20:54:22.430 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87168.log.gz 2026-02-22T20:54:22.430 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79341.log 2026-02-22T20:54:22.430 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97508.log.gz 2026-02-22T20:54:22.430 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64216.log.gz 2026-02-22T20:54:22.430 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61760.log 2026-02-22T20:54:22.430 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76110.log 2026-02-22T20:54:22.430 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79341.log.gz 2026-02-22T20:54:22.430 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61760.log.gz 2026-02-22T20:54:22.430 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103979.log 2026-02-22T20:54:22.430 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71746.log 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76110.log.gz 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103979.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103979.log.gz 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78726.log 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56128.log 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71746.log.gz 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78726.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.66164.log -- replaced with /var/log/ceph/ceph-client.admin.78726.log.gz 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr: 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55514.log 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56128.log.gz 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66164.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.66164.log.gz -5 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.79749.log 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65957.log 2026-02-22T20:54:22.431 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55514.log.gz 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79749.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79749.log.gz -5 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.84099.log 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65957.log.gz 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57356.log 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84099.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.53710.log -- replaced with /var/log/ceph/ceph-client.admin.84099.log.gz 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr: 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57356.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57356.log.gz 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82104.log 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108894.log 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53710.log.gz 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85941.log 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82104.log.gz 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108894.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.74255.log -- replaced with /var/log/ceph/ceph-client.admin.108894.log.gz 2026-02-22T20:54:22.432 INFO:teuthology.orchestra.run.trial084.stderr: 2026-02-22T20:54:22.433 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85941.log.gz 2026-02-22T20:54:22.433 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52074.log 2026-02-22T20:54:22.433 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54746.log 2026-02-22T20:54:22.433 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74255.log.gz 2026-02-22T20:54:22.433 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52074.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52074.log.gz 2026-02-22T20:54:22.433 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90854.log 2026-02-22T20:54:22.433 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60277.log 2026-02-22T20:54:22.433 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54746.log.gz 2026-02-22T20:54:22.433 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49204.log 2026-02-22T20:54:22.433 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90854.log.gz 2026-02-22T20:54:22.434 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60277.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.60277.log.gz 2026-02-22T20:54:22.434 INFO:teuthology.orchestra.run.trial084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48643.log 2026-02-22T20:54:22.434 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50181.log 2026-02-22T20:54:22.434 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49204.log.gz 2026-02-22T20:54:22.434 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48643.log.gz 2026-02-22T20:54:22.434 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89062.log 2026-02-22T20:54:22.434 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50181.log.gz 2026-02-22T20:54:22.434 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84560.log 2026-02-22T20:54:22.434 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89062.log.gzgzip 2026-02-22T20:54:22.434 INFO:teuthology.orchestra.run.trial084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56537.log 2026-02-22T20:54:22.435 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84560.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84560.log.gz 2026-02-22T20:54:22.435 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96842.log 2026-02-22T20:54:22.435 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106898.log 2026-02-22T20:54:22.435 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56537.log.gz 2026-02-22T20:54:22.435 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110589.log 2026-02-22T20:54:22.435 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96842.log.gz 2026-02-22T20:54:22.436 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71026.log 2026-02-22T20:54:22.436 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106898.log.gz 2026-02-22T20:54:22.436 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110589.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110589.log.gz 2026-02-22T20:54:22.436 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46699.log 2026-02-22T20:54:22.436 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71026.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71026.log.gz 2026-02-22T20:54:22.436 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110435.log 2026-02-22T20:54:22.436 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59252.log 2026-02-22T20:54:22.436 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46699.log.gz 2026-02-22T20:54:22.436 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110435.log.gz 2026-02-22T20:54:22.436 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67700.log 2026-02-22T20:54:22.437 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71388.log 2026-02-22T20:54:22.437 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59252.log.gz 2026-02-22T20:54:22.437 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88704.log 2026-02-22T20:54:22.437 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67700.log.gz 2026-02-22T20:54:22.437 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71388.log.gz 2026-02-22T20:54:22.437 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43642.log 2026-02-22T20:54:22.437 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88704.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88704.log.gzgzip 2026-02-22T20:54:22.437 INFO:teuthology.orchestra.run.trial084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62988.log 2026-02-22T20:54:22.438 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87477.log 2026-02-22T20:54:22.438 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43642.log.gz 2026-02-22T20:54:22.438 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62988.log.gz 2026-02-22T20:54:22.438 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52840.log 2026-02-22T20:54:22.438 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76673.log 2026-02-22T20:54:22.438 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87477.log.gz 2026-02-22T20:54:22.439 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52840.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52840.log.gz 2026-02-22T20:54:22.439 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98845.log 2026-02-22T20:54:22.439 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65138.log 2026-02-22T20:54:22.439 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76673.log.gz 2026-02-22T20:54:22.439 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91468.log 2026-02-22T20:54:22.439 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98845.log.gz 2026-02-22T20:54:22.439 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65138.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65138.log.gz 2026-02-22T20:54:22.439 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85584.log 2026-02-22T20:54:22.440 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59303.log 2026-02-22T20:54:22.440 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91468.log.gz 2026-02-22T20:54:22.440 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85584.log.gz 2026-02-22T20:54:22.440 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84252.log 2026-02-22T20:54:22.440 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59303.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59303.log.gz 2026-02-22T20:54:22.440 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94539.log 2026-02-22T20:54:22.440 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84252.log.gz 2026-02-22T20:54:22.440 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64882.log 2026-02-22T20:54:22.441 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55412.log 2026-02-22T20:54:22.441 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94539.log.gz 2026-02-22T20:54:22.441 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64882.log.gz 2026-02-22T20:54:22.441 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89933.log 2026-02-22T20:54:22.441 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109047.log 2026-02-22T20:54:22.441 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55412.log.gz 2026-02-22T20:54:22.441 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46236.log 2026-02-22T20:54:22.441 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89933.log.gz 2026-02-22T20:54:22.442 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109047.log.gz 2026-02-22T20:54:22.442 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96330.log 2026-02-22T20:54:22.442 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81489.log 2026-02-22T20:54:22.442 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46236.log.gz 2026-02-22T20:54:22.442 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96330.log.gz 2026-02-22T20:54:22.442 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69338.log 2026-02-22T20:54:22.443 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104594.log 2026-02-22T20:54:22.443 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81489.log.gz 2026-02-22T20:54:22.443 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109405.log 2026-02-22T20:54:22.443 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69338.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69338.log.gz 2026-02-22T20:54:22.443 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104594.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104594.log.gz 2026-02-22T20:54:22.443 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88398.log 2026-02-22T20:54:22.443 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109405.log.gzgzip 2026-02-22T20:54:22.443 INFO:teuthology.orchestra.run.trial084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.79953.log 2026-02-22T20:54:22.444 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46032.log 2026-02-22T20:54:22.444 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88398.log.gz 2026-02-22T20:54:22.444 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79953.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.79953.log.gz -5 2026-02-22T20:54:22.444 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55924.log 2026-02-22T20:54:22.444 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99938.log 2026-02-22T20:54:22.444 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46032.log.gz 2026-02-22T20:54:22.444 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55924.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55924.log.gz 2026-02-22T20:54:22.444 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103826.log 2026-02-22T20:54:22.445 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99938.log.gz 2026-02-22T20:54:22.445 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84456.log 2026-02-22T20:54:22.445 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103826.log.gz 2026-02-22T20:54:22.445 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57458.log 2026-02-22T20:54:22.445 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62374.log 2026-02-22T20:54:22.445 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84456.log.gz 2026-02-22T20:54:22.445 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57458.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57458.log.gz 2026-02-22T20:54:22.445 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64012.log 2026-02-22T20:54:22.446 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91111.log 2026-02-22T20:54:22.446 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62374.log.gz 2026-02-22T20:54:22.446 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107306.log 2026-02-22T20:54:22.446 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64012.log.gz 2026-02-22T20:54:22.446 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91111.log.gz 2026-02-22T20:54:22.446 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50385.log 2026-02-22T20:54:22.446 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93824.log 2026-02-22T20:54:22.446 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107306.log.gz 2026-02-22T20:54:22.446 INFO:teuthology.orchestra.run.trial084.stderr:gzip/var/log/ceph/ceph-client.admin.50385.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.105617.log 2026-02-22T20:54:22.447 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50385.log.gz 2026-02-22T20:54:22.447 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47926.log 2026-02-22T20:54:22.447 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93824.log.gz 2026-02-22T20:54:22.447 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75432.log 2026-02-22T20:54:22.447 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105617.log.gz 2026-02-22T20:54:22.447 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47926.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.47926.log.gz -- 2026-02-22T20:54:22.447 INFO:teuthology.orchestra.run.trial084.stderr: /var/log/ceph/ceph-client.admin.78267.log 2026-02-22T20:54:22.448 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75432.log.gz 2026-02-22T20:54:22.448 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46441.log 2026-02-22T20:54:22.448 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78267.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.78267.log.gz /var/log/ceph/ceph-client.admin.62630.log 2026-02-22T20:54:22.448 INFO:teuthology.orchestra.run.trial084.stderr: 2026-02-22T20:54:22.448 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104082.log 2026-02-22T20:54:22.448 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46441.log.gz 2026-02-22T20:54:22.448 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62630.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62630.log.gz 2026-02-22T20:54:22.448 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55565.log 2026-02-22T20:54:22.449 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104082.log.gz 2026-02-22T20:54:22.449 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48388.log 2026-02-22T20:54:22.449 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95666.log 2026-02-22T20:54:22.449 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55565.log.gz 2026-02-22T20:54:22.449 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48388.log.gz 2026-02-22T20:54:22.449 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107460.log 2026-02-22T20:54:22.450 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43292.log 2026-02-22T20:54:22.450 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95666.log.gz 2026-02-22T20:54:22.450 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107460.log.gz 2026-02-22T20:54:22.450 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97098.log 2026-02-22T20:54:22.450 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43292.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43292.log.gz 2026-02-22T20:54:22.450 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51408.log 2026-02-22T20:54:22.450 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64779.log 2026-02-22T20:54:22.450 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97098.log.gz 2026-02-22T20:54:22.450 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107972.log 2026-02-22T20:54:22.450 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51408.log.gz 2026-02-22T20:54:22.451 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97200.log 2026-02-22T20:54:22.451 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64779.log.gz 2026-02-22T20:54:22.451 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107972.log.gz 2026-02-22T20:54:22.451 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38652.log 2026-02-22T20:54:22.451 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68774.log 2026-02-22T20:54:22.451 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97200.log.gz 2026-02-22T20:54:22.452 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90394.log 2026-02-22T20:54:22.452 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.38652.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38652.log.gz 2026-02-22T20:54:22.452 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68774.log.gz 2026-02-22T20:54:22.452 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62936.log 2026-02-22T20:54:22.452 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92954.log 2026-02-22T20:54:22.452 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90394.log.gz 2026-02-22T20:54:22.452 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62936.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.62936.log.gz -5 2026-02-22T20:54:22.452 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.78675.log 2026-02-22T20:54:22.453 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43494.log 2026-02-22T20:54:22.453 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92954.log.gz 2026-02-22T20:54:22.453 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78675.log.gz 2026-02-22T20:54:22.453 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45981.log 2026-02-22T20:54:22.453 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44531.log 2026-02-22T20:54:22.453 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43494.log.gz 2026-02-22T20:54:22.453 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87015.log 2026-02-22T20:54:22.453 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45981.log.gz 2026-02-22T20:54:22.454 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48796.log 2026-02-22T20:54:22.454 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44531.log.gz 2026-02-22T20:54:22.454 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87015.log.gz 2026-02-22T20:54:22.454 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87630.log 2026-02-22T20:54:22.454 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48796.log.gz 2026-02-22T20:54:22.454 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110025.log 2026-02-22T20:54:22.454 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81285.log 2026-02-22T20:54:22.454 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87630.log.gz 2026-02-22T20:54:22.455 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89626.log 2026-02-22T20:54:22.455 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110025.log.gz 2026-02-22T20:54:22.455 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81285.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81285.log.gz 2026-02-22T20:54:22.455 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76828.log 2026-02-22T20:54:22.455 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83997.log 2026-02-22T20:54:22.455 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89626.log.gz 2026-02-22T20:54:22.455 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84918.log 2026-02-22T20:54:22.455 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76828.log.gz 2026-02-22T20:54:22.456 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83997.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83997.log.gz -5 2026-02-22T20:54:22.456 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.104440.log 2026-02-22T20:54:22.456 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42571.log 2026-02-22T20:54:22.456 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84918.log.gz 2026-02-22T20:54:22.456 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79902.log 2026-02-22T20:54:22.456 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104440.log.gz 2026-02-22T20:54:22.456 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42571.log.gz 2026-02-22T20:54:22.456 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97980.log 2026-02-22T20:54:22.457 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55258.log 2026-02-22T20:54:22.457 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79902.log.gz 2026-02-22T20:54:22.457 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73333.log 2026-02-22T20:54:22.457 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97980.log.gz 2026-02-22T20:54:22.457 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51765.log 2026-02-22T20:54:22.457 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55258.log.gz 2026-02-22T20:54:22.458 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97763.log 2026-02-22T20:54:22.458 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73333.log.gz 2026-02-22T20:54:22.458 INFO:teuthology.orchestra.run.trial084.stderr:gzip/var/log/ceph/ceph-client.admin.51765.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47311.log 2026-02-22T20:54:22.458 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51765.log.gz 2026-02-22T20:54:22.458 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68262.log 2026-02-22T20:54:22.458 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97763.log.gz 2026-02-22T20:54:22.458 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47311.log.gz 2026-02-22T20:54:22.458 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80466.log 2026-02-22T20:54:22.459 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69492.log 2026-02-22T20:54:22.459 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68262.log.gz 2026-02-22T20:54:22.459 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95972.log 2026-02-22T20:54:22.459 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80466.log.gz 2026-02-22T20:54:22.459 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81132.log 2026-02-22T20:54:22.459 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69492.log.gz 2026-02-22T20:54:22.459 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95972.log.gz 2026-02-22T20:54:22.459 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90139.log 2026-02-22T20:54:22.460 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78777.log 2026-02-22T20:54:22.460 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81132.log.gz 2026-02-22T20:54:22.460 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99581.log 2026-02-22T20:54:22.460 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90139.log.gz 2026-02-22T20:54:22.460 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78777.log.gz 2026-02-22T20:54:22.460 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90905.log 2026-02-22T20:54:22.460 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42189.log 2026-02-22T20:54:22.460 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99581.log.gz 2026-02-22T20:54:22.461 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90905.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90905.log.gz 2026-02-22T20:54:22.461 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97406.log 2026-02-22T20:54:22.461 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42189.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42189.log.gz -5 2026-02-22T20:54:22.461 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.79290.log 2026-02-22T20:54:22.461 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82872.log 2026-02-22T20:54:22.461 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97406.log.gz 2026-02-22T20:54:22.461 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73282.log 0.0% 2026-02-22T20:54:22.461 INFO:teuthology.orchestra.run.trial084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79290.log.gz 2026-02-22T20:54:22.462 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86862.log 2026-02-22T20:54:22.462 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82872.log.gz 2026-02-22T20:54:22.462 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45828.log 2026-02-22T20:54:22.462 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73282.log.gz 2026-02-22T20:54:22.462 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49258.log 2026-02-22T20:54:22.462 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86862.log.gz 2026-02-22T20:54:22.462 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69285.log 2026-02-22T20:54:22.463 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45828.log.gz 2026-02-22T20:54:22.463 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49258.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49258.log.gz 2026-02-22T20:54:22.463 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72973.log 2026-02-22T20:54:22.463 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83127.log 2026-02-22T20:54:22.463 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69285.log.gz 2026-02-22T20:54:22.463 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76518.log 2026-02-22T20:54:22.463 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72973.log.gz 2026-02-22T20:54:22.464 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50232.log 2026-02-22T20:54:22.464 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83127.log.gz 2026-02-22T20:54:22.464 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59764.log 2026-02-22T20:54:22.464 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76518.log.gz 2026-02-22T20:54:22.464 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72820.log 2026-02-22T20:54:22.464 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50232.log.gz 2026-02-22T20:54:22.464 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59764.log.gz 2026-02-22T20:54:22.464 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77958.log 2026-02-22T20:54:22.465 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72820.log.gz 2026-02-22T20:54:22.465 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50283.log 2026-02-22T20:54:22.465 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99075.log 2026-02-22T20:54:22.465 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77958.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77958.log.gz 2026-02-22T20:54:22.465 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52278.log 2026-02-22T20:54:22.465 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50283.log.gz 2026-02-22T20:54:22.465 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99075.log.gz 2026-02-22T20:54:22.465 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108792.log 2026-02-22T20:54:22.466 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52278.log.gz 2026-02-22T20:54:22.466 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97929.log 2026-02-22T20:54:22.466 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100289.log 2026-02-22T20:54:22.466 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108792.log.gz 2026-02-22T20:54:22.466 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97929.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97929.log.gz 2026-02-22T20:54:22.466 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48694.log 2026-02-22T20:54:22.466 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56435.log 2026-02-22T20:54:22.466 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100289.log.gz 2026-02-22T20:54:22.467 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65598.log 2026-02-22T20:54:22.467 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48694.log.gz 2026-02-22T20:54:22.467 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47569.log 2026-02-22T20:54:22.467 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56435.log.gz 2026-02-22T20:54:22.467 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93107.log 2026-02-22T20:54:22.467 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65598.log.gz 2026-02-22T20:54:22.467 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47569.log.gz 2026-02-22T20:54:22.467 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109201.log 2026-02-22T20:54:22.468 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105361.log 2026-02-22T20:54:22.468 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93107.log.gz 2026-02-22T20:54:22.468 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109201.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109201.log.gz 2026-02-22T20:54:22.468 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43153.log 2026-02-22T20:54:22.468 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90190.log 2026-02-22T20:54:22.468 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105361.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105361.log.gz 2026-02-22T20:54:22.468 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43153.log.gz 2026-02-22T20:54:22.468 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76314.log 2026-02-22T20:54:22.469 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90190.log.gz 2026-02-22T20:54:22.469 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98444.log 2026-02-22T20:54:22.469 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91674.log 2026-02-22T20:54:22.469 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76314.log.gz 2026-02-22T20:54:22.469 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108688.log 2026-02-22T20:54:22.469 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98444.log.gz 2026-02-22T20:54:22.470 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100669.log 2026-02-22T20:54:22.470 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91674.log.gz 2026-02-22T20:54:22.470 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108688.log.gz 2026-02-22T20:54:22.470 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103161.log 2026-02-22T20:54:22.470 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62834.log 2026-02-22T20:54:22.470 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100669.log.gz 2026-02-22T20:54:22.470 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44298.log 2026-02-22T20:54:22.470 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103161.log.gz 2026-02-22T20:54:22.470 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62834.log.gz 2026-02-22T20:54:22.470 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107153.log 2026-02-22T20:54:22.471 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78163.log 2026-02-22T20:54:22.471 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44298.log.gz 2026-02-22T20:54:22.471 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80109.log 2026-02-22T20:54:22.471 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107153.log.gz 2026-02-22T20:54:22.471 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49051.log 2026-02-22T20:54:22.471 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78163.log.gz 2026-02-22T20:54:22.471 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80109.log.gz 2026-02-22T20:54:22.471 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69081.log 2026-02-22T20:54:22.472 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49051.log.gz 2026-02-22T20:54:22.472 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95205.log 2026-02-22T20:54:22.472 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79851.log 2026-02-22T20:54:22.472 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69081.log.gz 2026-02-22T20:54:22.472 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59918.log 2026-02-22T20:54:22.472 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95205.log.gz 2026-02-22T20:54:22.472 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59560.log 2026-02-22T20:54:22.472 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79851.log.gz 2026-02-22T20:54:22.473 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43050.log 2026-02-22T20:54:22.473 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59918.log.gz 2026-02-22T20:54:22.473 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68979.log 2026-02-22T20:54:22.473 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59560.log.gz 2026-02-22T20:54:22.473 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72514.log 2026-02-22T20:54:22.473 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43050.log.gz 2026-02-22T20:54:22.473 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91366.log 2026-02-22T20:54:22.473 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68979.log.gz 2026-02-22T20:54:22.474 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72514.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72514.log.gz -5 2026-02-22T20:54:22.474 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49770.log 2026-02-22T20:54:22.474 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58280.log 2026-02-22T20:54:22.474 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91366.log.gz 2026-02-22T20:54:22.474 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72922.log 2026-02-22T20:54:22.474 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49770.log.gz 2026-02-22T20:54:22.474 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58280.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58280.log.gz 2026-02-22T20:54:22.475 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49102.log 2026-02-22T20:54:22.475 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72922.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72922.log.gz 2026-02-22T20:54:22.475 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64369.log 2026-02-22T20:54:22.475 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43699.log 2026-02-22T20:54:22.475 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49102.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49102.log.gz 2026-02-22T20:54:22.475 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97149.log 2026-02-22T20:54:22.475 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64369.log.gz 2026-02-22T20:54:22.475 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43699.log.gz 2026-02-22T20:54:22.476 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92288.log 2026-02-22T20:54:22.476 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54850.log 2026-02-22T20:54:22.476 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97149.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97149.log.gz 2026-02-22T20:54:22.476 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91776.log 2026-02-22T20:54:22.476 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92288.log.gz 2026-02-22T20:54:22.476 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54850.log.gz 2026-02-22T20:54:22.476 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62323.log 2026-02-22T20:54:22.476 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91776.log.gz 2026-02-22T20:54:22.476 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99351.log 2026-02-22T20:54:22.477 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62323.log.gz 2026-02-22T20:54:22.477 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77442.log 2026-02-22T20:54:22.477 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47671.log 2026-02-22T20:54:22.477 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99351.log.gz 2026-02-22T20:54:22.477 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.77442.log: /var/log/ceph/ceph-client.admin.52686.log 2026-02-22T20:54:22.477 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77442.log.gz 2026-02-22T20:54:22.477 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47671.log.gz 2026-02-22T20:54:22.477 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108637.log 2026-02-22T20:54:22.478 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55003.log 2026-02-22T20:54:22.478 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52686.log.gz 2026-02-22T20:54:22.478 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82821.log 2026-02-22T20:54:22.478 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108637.log.gz 2026-02-22T20:54:22.478 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55003.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55003.log.gz 2026-02-22T20:54:22.478 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46288.log 2026-02-22T20:54:22.478 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44400.log 2026-02-22T20:54:22.478 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82821.log.gz 2026-02-22T20:54:22.479 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46288.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46288.log.gz 2026-02-22T20:54:22.479 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88038.log 2026-02-22T20:54:22.479 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72052.log 2026-02-22T20:54:22.479 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44400.log.gz 2026-02-22T20:54:22.479 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51510.log 2026-02-22T20:54:22.479 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88038.log.gz 2026-02-22T20:54:22.479 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62425.log 2026-02-22T20:54:22.480 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72052.log.gz 2026-02-22T20:54:22.480 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51510.log.gz 2026-02-22T20:54:22.480 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51049.log 2026-02-22T20:54:22.480 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105208.log 2026-02-22T20:54:22.480 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62425.log.gz 2026-02-22T20:54:22.480 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99121.log 2026-02-22T20:54:22.480 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51049.log.gz 2026-02-22T20:54:22.480 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66677.log 2026-02-22T20:54:22.480 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105208.log.gz 2026-02-22T20:54:22.481 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99121.log.gz 2026-02-22T20:54:22.481 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50589.log 2026-02-22T20:54:22.481 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55207.log 2026-02-22T20:54:22.481 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66677.log.gz 2026-02-22T20:54:22.481 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110179.log 2026-02-22T20:54:22.481 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50589.log.gz 2026-02-22T20:54:22.481 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62885.log 2026-02-22T20:54:22.482 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55207.log.gz 2026-02-22T20:54:22.482 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86811.log 2026-02-22T20:54:22.482 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110179.log.gz 2026-02-22T20:54:22.482 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86454.log 2026-02-22T20:54:22.482 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62885.log.gz 2026-02-22T20:54:22.482 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86811.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86811.log.gz 2026-02-22T20:54:22.482 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46852.log 2026-02-22T20:54:22.483 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97354.log 2026-02-22T20:54:22.483 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86454.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86454.log.gz 2026-02-22T20:54:22.483 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46852.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.46852.log.gz --verbose 2026-02-22T20:54:22.483 INFO:teuthology.orchestra.run.trial084.stderr: -- /var/log/ceph/ceph-client.admin.74460.log 2026-02-22T20:54:22.483 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58690.log 2026-02-22T20:54:22.483 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97354.log.gz 2026-02-22T20:54:22.483 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67494.log 2026-02-22T20:54:22.484 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74460.log.gz 2026-02-22T20:54:22.484 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58690.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58690.log.gz 2026-02-22T20:54:22.484 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47722.log 2026-02-22T20:54:22.484 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107870.log 2026-02-22T20:54:22.484 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67494.log.gz 2026-02-22T20:54:22.484 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47722.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47722.log.gz 2026-02-22T20:54:22.484 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69234.log 2026-02-22T20:54:22.484 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107870.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107870.log.gz 2026-02-22T20:54:22.485 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81693.log 2026-02-22T20:54:22.485 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84714.log 2026-02-22T20:54:22.485 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69234.log.gz 2026-02-22T20:54:22.485 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81693.log.gz 2026-02-22T20:54:22.485 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78879.log 2026-02-22T20:54:22.485 INFO:teuthology.orchestra.run.trial084.stderr:gzip/var/log/ceph/ceph-client.admin.84714.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38310.log 2026-02-22T20:54:22.485 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84714.log.gz 2026-02-22T20:54:22.486 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63653.log 2026-02-22T20:54:22.486 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78879.log.gz 2026-02-22T20:54:22.486 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.38310.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38310.log.gz 2026-02-22T20:54:22.486 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70105.log 2026-02-22T20:54:22.486 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104030.log 2026-02-22T20:54:22.486 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63653.log.gz 2026-02-22T20:54:22.486 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70105.log.gz 2026-02-22T20:54:22.486 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108176.log 2026-02-22T20:54:22.487 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52995.log 2026-02-22T20:54:22.487 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104030.log.gz 2026-02-22T20:54:22.487 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108176.log.gz 2026-02-22T20:54:22.487 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50946.log 2026-02-22T20:54:22.487 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58792.log 2026-02-22T20:54:22.487 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52995.log.gz 2026-02-22T20:54:22.487 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42572.log 2026-02-22T20:54:22.487 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50946.log.gz 2026-02-22T20:54:22.488 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82974.log 2026-02-22T20:54:22.488 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58792.log.gz 2026-02-22T20:54:22.488 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45063.log 2026-02-22T20:54:22.488 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42572.log.gz 2026-02-22T20:54:22.488 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82974.log.gz 2026-02-22T20:54:22.488 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68211.log 2026-02-22T20:54:22.488 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83638.log 2026-02-22T20:54:22.488 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45063.log.gz 2026-02-22T20:54:22.489 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47260.log 2026-02-22T20:54:22.489 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68211.log.gz 2026-02-22T20:54:22.489 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83638.log.gz 2026-02-22T20:54:22.489 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93926.log 2026-02-22T20:54:22.489 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49515.log 2026-02-22T20:54:22.489 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47260.log.gz 2026-02-22T20:54:22.490 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93926.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93926.log.gz 2026-02-22T20:54:22.490 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89269.log 2026-02-22T20:54:22.490 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108074.log 2026-02-22T20:54:22.490 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49515.log.gz 2026-02-22T20:54:22.490 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89269.log.gz 2026-02-22T20:54:22.490 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83535.log 2026-02-22T20:54:22.490 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96126.log 2026-02-22T20:54:22.491 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108074.log.gz 2026-02-22T20:54:22.491 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83535.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83535.log.gz 2026-02-22T20:54:22.491 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91060.log 2026-02-22T20:54:22.491 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96126.log.gz 2026-02-22T20:54:22.491 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58382.log 2026-02-22T20:54:22.491 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108330.log 2026-02-22T20:54:22.491 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91060.log.gz 2026-02-22T20:54:22.492 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58382.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.103109.log 2026-02-22T20:54:22.492 INFO:teuthology.orchestra.run.trial084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58382.log.gz 2026-02-22T20:54:22.492 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58537.log 2026-02-22T20:54:22.492 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108330.log.gz 2026-02-22T20:54:22.492 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103109.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.94386.log 2026-02-22T20:54:22.492 INFO:teuthology.orchestra.run.trial084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103109.log.gz 2026-02-22T20:54:22.492 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81900.log 2026-02-22T20:54:22.492 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58537.log.gz 2026-02-22T20:54:22.493 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59201.log 2026-02-22T20:54:22.493 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94386.log.gz 2026-02-22T20:54:22.493 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69849.log 2026-02-22T20:54:22.493 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81900.log.gz 2026-02-22T20:54:22.493 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63040.log 2026-02-22T20:54:22.493 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59201.log.gz 2026-02-22T20:54:22.493 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69849.log.gz 2026-02-22T20:54:22.493 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53250.log 2026-02-22T20:54:22.494 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5/var/log/ceph/ceph-client.admin.63040.log: --verbose -- /var/log/ceph/ceph-client.admin.103264.log 2026-02-22T20:54:22.494 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63040.log.gz 2026-02-22T20:54:22.494 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43817.log 2026-02-22T20:54:22.494 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53250.log.gz 2026-02-22T20:54:22.494 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103264.log.gz 2026-02-22T20:54:22.494 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77499.log 2026-02-22T20:54:22.495 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43817.log.gz 2026-02-22T20:54:22.495 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103724.log 2026-02-22T20:54:22.495 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91264.log 2026-02-22T20:54:22.495 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77499.log.gz 2026-02-22T20:54:22.495 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103724.log.gz 2026-02-22T20:54:22.495 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99627.log 2026-02-22T20:54:22.495 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48745.log 2026-02-22T20:54:22.495 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91264.log.gz 2026-02-22T20:54:22.496 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99627.log.gz 2026-02-22T20:54:22.496 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108432.log 2026-02-22T20:54:22.496 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62732.log 2026-02-22T20:54:22.496 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48745.log.gz 2026-02-22T20:54:22.496 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90343.log 2026-02-22T20:54:22.496 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108432.log.gz 2026-02-22T20:54:22.496 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50487.log 2026-02-22T20:54:22.496 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62732.log.gz 2026-02-22T20:54:22.497 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72360.log 2026-02-22T20:54:22.497 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90343.log.gz 2026-02-22T20:54:22.497 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54132.log 2026-02-22T20:54:22.497 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50487.log.gz 2026-02-22T20:54:22.497 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72360.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72360.log.gz 2026-02-22T20:54:22.497 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90752.log 2026-02-22T20:54:22.497 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85839.log 2026-02-22T20:54:22.497 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54132.log.gz 2026-02-22T20:54:22.498 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47158.log 2026-02-22T20:54:22.498 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90752.log.gz 2026-02-22T20:54:22.498 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70566.log 2026-02-22T20:54:22.498 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85839.log.gz 2026-02-22T20:54:22.498 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73844.log 2026-02-22T20:54:22.498 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47158.log.gz 2026-02-22T20:54:22.498 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55054.log 2026-02-22T20:54:22.498 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70566.log.gz 2026-02-22T20:54:22.499 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88806.log 2026-02-22T20:54:22.499 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73844.log.gz 2026-02-22T20:54:22.499 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70771.log 2026-02-22T20:54:22.499 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55054.log.gz 2026-02-22T20:54:22.499 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77390.log 2026-02-22T20:54:22.499 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88806.log.gz 2026-02-22T20:54:22.499 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70771.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70771.log.gz 2026-02-22T20:54:22.499 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56077.log 2026-02-22T20:54:22.500 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77390.log.gz 2026-02-22T20:54:22.500 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93414.log 2026-02-22T20:54:22.500 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75228.log 2026-02-22T20:54:22.500 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56077.log.gz 2026-02-22T20:54:22.500 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91571.log 2026-02-22T20:54:22.500 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93414.log.gz 2026-02-22T20:54:22.500 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98133.log 2026-02-22T20:54:22.501 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75228.log.gz 2026-02-22T20:54:22.501 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91571.log.gz 2026-02-22T20:54:22.501 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63142.log 2026-02-22T20:54:22.501 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47467.log 2026-02-22T20:54:22.501 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98133.log.gz 2026-02-22T20:54:22.501 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71848.log 2026-02-22T20:54:22.501 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63142.log.gz 2026-02-22T20:54:22.501 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74101.log 2026-02-22T20:54:22.501 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47467.log.gz 2026-02-22T20:54:22.502 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80517.log 2026-02-22T20:54:22.502 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71848.log.gz 2026-02-22T20:54:22.502 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74101.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.74101.log.gz --verbose 2026-02-22T20:54:22.502 INFO:teuthology.orchestra.run.trial084.stderr: -- /var/log/ceph/ceph-client.admin.73999.log 2026-02-22T20:54:22.503 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80517.log.gz 2026-02-22T20:54:22.503 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89728.log 2026-02-22T20:54:22.503 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63602.log 2026-02-22T20:54:22.503 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73999.log.gz 2026-02-22T20:54:22.503 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89728.log.gz 2026-02-22T20:54:22.504 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49464.log 2026-02-22T20:54:22.504 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63397.log 2026-02-22T20:54:22.504 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63602.log.gz 2026-02-22T20:54:22.504 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49464.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.49464.log.gz /var/log/ceph/ceph-client.admin.44179.log 2026-02-22T20:54:22.504 INFO:teuthology.orchestra.run.trial084.stderr: 2026-02-22T20:54:22.504 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103877.log 2026-02-22T20:54:22.504 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63397.log.gz 2026-02-22T20:54:22.504 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47977.log 2026-02-22T20:54:22.505 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44179.log.gz 2026-02-22T20:54:22.505 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105975.log 2026-02-22T20:54:22.505 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103877.log.gz 2026-02-22T20:54:22.505 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47977.log.gz 2026-02-22T20:54:22.505 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107409.log 2026-02-22T20:54:22.505 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77907.log 2026-02-22T20:54:22.505 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105975.log.gz 2026-02-22T20:54:22.505 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107409.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107409.log.gz 2026-02-22T20:54:22.505 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49821.log 2026-02-22T20:54:22.506 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77907.log.gz 2026-02-22T20:54:22.506 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54438.log 2026-02-22T20:54:22.506 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93977.log 2026-02-22T20:54:22.506 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49821.log.gz 2026-02-22T20:54:22.506 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54438.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54438.log.gz 2026-02-22T20:54:22.506 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87681.log 2026-02-22T20:54:22.507 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93977.log.gz 2026-02-22T20:54:22.507 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105822.log 2026-02-22T20:54:22.507 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77135.log 2026-02-22T20:54:22.507 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87681.log.gz 2026-02-22T20:54:22.507 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42115.log 2026-02-22T20:54:22.507 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105822.log.gz 2026-02-22T20:54:22.508 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57305.log 2026-02-22T20:54:22.508 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77135.log.gz 2026-02-22T20:54:22.508 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42115.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-22T20:54:22.508 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42115.log.gz 2026-02-22T20:54:22.508 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57305.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.57305.log.gz -- 2026-02-22T20:54:22.508 INFO:teuthology.orchestra.run.trial084.stderr: /var/log/ceph/ceph-client.admin.100954.log 2026-02-22T20:54:22.508 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66626.log 2026-02-22T20:54:22.508 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92799.log 2026-02-22T20:54:22.509 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100954.log.gz 2026-02-22T20:54:22.509 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66626.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.66626.log.gz --verbose -- 2026-02-22T20:54:22.509 INFO:teuthology.orchestra.run.trial084.stderr: /var/log/ceph/ceph-client.admin.50538.log 2026-02-22T20:54:22.509 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92799.log.gz 2026-02-22T20:54:22.509 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105157.log 2026-02-22T20:54:22.509 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89524.log 2026-02-22T20:54:22.509 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50538.log.gz 2026-02-22T20:54:22.509 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105157.log.gz 2026-02-22T20:54:22.509 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90649.log 2026-02-22T20:54:22.510 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94130.log 2026-02-22T20:54:22.510 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89524.log.gz 2026-02-22T20:54:22.510 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90649.log.gz 2026-02-22T20:54:22.510 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81540.log 2026-02-22T20:54:22.510 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49719.log 2026-02-22T20:54:22.510 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94130.log.gz 2026-02-22T20:54:22.511 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81540.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81540.log.gz 2026-02-22T20:54:22.511 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99489.log 2026-02-22T20:54:22.511 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74972.log 2026-02-22T20:54:22.511 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49719.log.gz 2026-02-22T20:54:22.511 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99489.log.gz 2026-02-22T20:54:22.511 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56384.log 2026-02-22T20:54:22.511 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47824.log 2026-02-22T20:54:22.511 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74972.log.gz 2026-02-22T20:54:22.512 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56384.log.gz 2026-02-22T20:54:22.512 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75381.log 2026-02-22T20:54:22.512 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47824.log.gz 2026-02-22T20:54:22.512 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38009.log 2026-02-22T20:54:22.512 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57919.log 2026-02-22T20:54:22.512 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75381.log.gz 2026-02-22T20:54:22.512 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.38009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38009.log.gz 2026-02-22T20:54:22.512 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71695.log 2026-02-22T20:54:22.513 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80619.log 2026-02-22T20:54:22.513 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57919.log.gz 2026-02-22T20:54:22.513 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75177.log 2026-02-22T20:54:22.513 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71695.log.gz 2026-02-22T20:54:22.513 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80619.log.gz 2026-02-22T20:54:22.513 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103928.log 2026-02-22T20:54:22.513 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48898.log 2026-02-22T20:54:22.513 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75177.log.gz 2026-02-22T20:54:22.514 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103928.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103928.log.gz 2026-02-22T20:54:22.514 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42778.log 2026-02-22T20:54:22.514 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68058.log 2026-02-22T20:54:22.514 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48898.log.gz 2026-02-22T20:54:22.514 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42778.log.gz 2026-02-22T20:54:22.514 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64421.log 2026-02-22T20:54:22.514 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89575.log 2026-02-22T20:54:22.515 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68058.log.gz 2026-02-22T20:54:22.515 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64421.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64421.log.gz 2026-02-22T20:54:22.515 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47107.log 2026-02-22T20:54:22.515 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89575.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2026-02-22T20:54:22.515 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89575.log.gz 2026-02-22T20:54:22.515 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71490.log 2026-02-22T20:54:22.515 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47107.log.gz 2026-02-22T20:54:22.515 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108996.log 2026-02-22T20:54:22.516 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59815.log 2026-02-22T20:54:22.516 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71490.log.gz 2026-02-22T20:54:22.516 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55873.log 2026-02-22T20:54:22.516 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108996.log.gz 2026-02-22T20:54:22.516 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69389.log 0.0% 2026-02-22T20:54:22.516 INFO:teuthology.orchestra.run.trial084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59815.log.gz 2026-02-22T20:54:22.517 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73436.log 2026-02-22T20:54:22.517 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55873.log.gz 2026-02-22T20:54:22.517 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45369.log 2026-02-22T20:54:22.517 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69389.log.gz 2026-02-22T20:54:22.517 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73436.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73436.log.gz 2026-02-22T20:54:22.517 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66216.log 2026-02-22T20:54:22.517 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44604.log 2026-02-22T20:54:22.517 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45369.log.gz 2026-02-22T20:54:22.518 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79137.log 2026-02-22T20:54:22.518 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66216.log.gz 2026-02-22T20:54:22.518 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44604.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.61964.log -- replaced with /var/log/ceph/ceph-client.admin.44604.log.gz 2026-02-22T20:54:22.518 INFO:teuthology.orchestra.run.trial084.stderr: 2026-02-22T20:54:22.518 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105924.log 2026-02-22T20:54:22.518 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79137.log.gz 2026-02-22T20:54:22.519 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53148.log 0.0% 2026-02-22T20:54:22.519 INFO:teuthology.orchestra.run.trial084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61964.log.gz 2026-02-22T20:54:22.519 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105924.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.93465.log -- replaced with /var/log/ceph/ceph-client.admin.105924.log.gz 2026-02-22T20:54:22.519 INFO:teuthology.orchestra.run.trial084.stderr: 2026-02-22T20:54:22.519 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52023.log 2026-02-22T20:54:22.519 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53148.log.gz 2026-02-22T20:54:22.520 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43102.log 2026-02-22T20:54:22.520 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93465.log.gz 2026-02-22T20:54:22.520 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52023.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52023.log.gz 2026-02-22T20:54:22.520 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45114.log 2026-02-22T20:54:22.520 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60992.log 2026-02-22T20:54:22.520 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43102.log.gz 2026-02-22T20:54:22.520 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45114.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.93209.log -- replaced with /var/log/ceph/ceph-client.admin.45114.log.gz 2026-02-22T20:54:22.520 INFO:teuthology.orchestra.run.trial084.stderr: 2026-02-22T20:54:22.521 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60992.log.gz 2026-02-22T20:54:22.521 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72105.log 2026-02-22T20:54:22.521 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93209.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93209.log.gz 2026-02-22T20:54:22.521 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56588.log 2026-02-22T20:54:22.521 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82717.log 2026-02-22T20:54:22.521 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72105.log.gz 2026-02-22T20:54:22.521 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56588.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56588.log.gz 2026-02-22T20:54:22.521 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58178.log 2026-02-22T20:54:22.522 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82717.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82717.log.gz 2026-02-22T20:54:22.522 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90035.log 2026-02-22T20:54:22.522 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58178.log.gz 2026-02-22T20:54:22.522 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82257.log 2026-02-22T20:54:22.522 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106384.log 2026-02-22T20:54:22.522 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90035.log.gz 2026-02-22T20:54:22.523 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66728.log 2026-02-22T20:54:22.523 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82257.log.gz 2026-02-22T20:54:22.523 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97457.log 2026-02-22T20:54:22.523 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106384.log.gz 2026-02-22T20:54:22.523 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68723.log 2026-02-22T20:54:22.523 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66728.log.gz 2026-02-22T20:54:22.524 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97457.log.gz 2026-02-22T20:54:22.524 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82563.log 2026-02-22T20:54:22.524 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66983.log 2026-02-22T20:54:22.524 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68723.log.gz 2026-02-22T20:54:22.524 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82563.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.82563.log.gz -5 2026-02-22T20:54:22.524 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48028.log 2026-02-22T20:54:22.524 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76365.log 2026-02-22T20:54:22.524 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66983.log.gz 2026-02-22T20:54:22.525 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99397.log 2026-02-22T20:54:22.525 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48028.log.gz 2026-02-22T20:54:22.525 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93158.log 2026-02-22T20:54:22.525 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76365.log.gz 2026-02-22T20:54:22.525 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75842.log 2026-02-22T20:54:22.525 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99397.log.gz 2026-02-22T20:54:22.525 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70822.log 2026-02-22T20:54:22.526 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93158.log.gz 2026-02-22T20:54:22.526 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75842.log.gz 2026-02-22T20:54:22.526 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61043.log 2026-02-22T20:54:22.526 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104185.log 2026-02-22T20:54:22.526 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70822.log.gz 2026-02-22T20:54:22.526 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65240.log 2026-02-22T20:54:22.526 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61043.log.gz 2026-02-22T20:54:22.526 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87375.log 2026-02-22T20:54:22.526 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104185.log.gz 2026-02-22T20:54:22.527 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54799.log 2026-02-22T20:54:22.527 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65240.log.gz 2026-02-22T20:54:22.527 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90241.log 2026-02-22T20:54:22.527 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87375.log.gz 2026-02-22T20:54:22.527 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54799.log.gz 2026-02-22T20:54:22.527 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72156.log 2026-02-22T20:54:22.528 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77601.log 2026-02-22T20:54:22.528 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90241.log.gz 2026-02-22T20:54:22.528 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86299.log 2026-02-22T20:54:22.528 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72156.log.gz 2026-02-22T20:54:22.528 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77601.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.77601.log.gz --verbose 2026-02-22T20:54:22.528 INFO:teuthology.orchestra.run.trial084.stderr: -- /var/log/ceph/ceph-client.admin.94898.log 2026-02-22T20:54:22.528 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109150.log 2026-02-22T20:54:22.528 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86299.log.gz 2026-02-22T20:54:22.529 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60122.log 2026-02-22T20:54:22.529 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94898.log.gz 2026-02-22T20:54:22.529 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73129.log 2026-02-22T20:54:22.529 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109150.log.gz 2026-02-22T20:54:22.529 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51102.log 2026-02-22T20:54:22.529 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60122.log.gz 2026-02-22T20:54:22.530 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73129.log.gz 2026-02-22T20:54:22.530 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80058.log 2026-02-22T20:54:22.530 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98650.log 2026-02-22T20:54:22.530 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51102.log.gz 2026-02-22T20:54:22.530 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69901.log 2026-02-22T20:54:22.530 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80058.log.gz 2026-02-22T20:54:22.530 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105310.log 2026-02-22T20:54:22.530 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98650.log.gz 2026-02-22T20:54:22.530 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68313.log 2026-02-22T20:54:22.530 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69901.log.gz 2026-02-22T20:54:22.531 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51868.log 2026-02-22T20:54:22.531 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105310.log.gz 2026-02-22T20:54:22.531 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45573.log 2026-02-22T20:54:22.531 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68313.log.gz 2026-02-22T20:54:22.531 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85890.log 2026-02-22T20:54:22.531 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51868.log.gz 2026-02-22T20:54:22.531 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98891.log 2026-02-22T20:54:22.531 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45573.log.gz 2026-02-22T20:54:22.532 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73385.log 2026-02-22T20:54:22.532 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85890.log.gz 2026-02-22T20:54:22.532 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98891.log.gz 2026-02-22T20:54:22.532 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91827.log 2026-02-22T20:54:22.532 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93005.log 2026-02-22T20:54:22.532 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73385.log.gz 2026-02-22T20:54:22.532 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85071.log 2026-02-22T20:54:22.532 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91827.log.gz 2026-02-22T20:54:22.533 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.93005.log: /var/log/ceph/ceph-client.admin.64114.log 2026-02-22T20:54:22.533 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93005.log.gz 2026-02-22T20:54:22.533 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78573.log 2026-02-22T20:54:22.533 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85071.log.gz 2026-02-22T20:54:22.533 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91931.log 2026-02-22T20:54:22.533 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64114.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64114.log.gz 2026-02-22T20:54:22.533 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58945.log 2026-02-22T20:54:22.533 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78573.log.gz 2026-02-22T20:54:22.534 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91931.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.91931.log.gz 2026-02-22T20:54:22.534 INFO:teuthology.orchestra.run.trial084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.94284.log 2026-02-22T20:54:22.534 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58945.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.58945.log.gz --verbose 2026-02-22T20:54:22.534 INFO:teuthology.orchestra.run.trial084.stderr: -- /var/log/ceph/ceph-client.admin.110383.log 2026-02-22T20:54:22.534 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92493.log 2026-02-22T20:54:22.534 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94284.log.gz 2026-02-22T20:54:22.534 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110383.log.gz 2026-02-22T20:54:22.534 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82410.log 2026-02-22T20:54:22.535 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103519.log 2026-02-22T20:54:22.535 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92493.log.gz 2026-02-22T20:54:22.535 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82410.log.gz 2026-02-22T20:54:22.535 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80568.log 2026-02-22T20:54:22.535 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108381.log 2026-02-22T20:54:22.535 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103519.log.gz 2026-02-22T20:54:22.535 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80568.log.gz 2026-02-22T20:54:22.535 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46597.log 2026-02-22T20:54:22.536 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81795.log 2026-02-22T20:54:22.536 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108381.log.gz 2026-02-22T20:54:22.536 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46597.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46597.log.gz 2026-02-22T20:54:22.536 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81849.log 2026-02-22T20:54:22.536 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58843.log 2026-02-22T20:54:22.536 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81795.log.gz 2026-02-22T20:54:22.536 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81849.log.gz 2026-02-22T20:54:22.536 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60532.log 2026-02-22T20:54:22.537 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100852.log 2026-02-22T20:54:22.537 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58843.log.gz 2026-02-22T20:54:22.537 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60532.log.gz 2026-02-22T20:54:22.537 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69952.log 2026-02-22T20:54:22.537 INFO:teuthology.orchestra.run.trial084.stderr:gzip/var/log/ceph/ceph-client.admin.100852.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.87885.log 2026-02-22T20:54:22.537 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100852.log.gz 2026-02-22T20:54:22.537 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69952.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69952.log.gz 2026-02-22T20:54:22.537 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106589.log 2026-02-22T20:54:22.538 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82614.log 2026-02-22T20:54:22.538 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87885.log.gz 2026-02-22T20:54:22.538 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106589.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106589.log.gz 2026-02-22T20:54:22.538 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43868.log 2026-02-22T20:54:22.538 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90088.log 2026-02-22T20:54:22.538 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82614.log.gz 2026-02-22T20:54:22.538 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43868.log.gz 2026-02-22T20:54:22.538 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92390.log 2026-02-22T20:54:22.539 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90088.log.gz 2026-02-22T20:54:22.539 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92748.log 2026-02-22T20:54:22.539 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2026-02-22T20:54:22.539 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92390.log.gz 2026-02-22T20:54:22.539 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92748.log.gz 2026-02-22T20:54:22.539 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98752.log 2026-02-22T20:54:22.540 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2026-02-22T20:54:22.540 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106231.log 2026-02-22T20:54:22.540 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98752.log.gz 2026-02-22T20:54:22.540 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68672.log 2026-02-22T20:54:22.540 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.106231.log: -- /var/log/ceph/ceph-client.admin.64523.log 2026-02-22T20:54:22.540 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106231.log.gz 2026-02-22T20:54:22.540 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68672.log.gz 2026-02-22T20:54:22.541 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79800.log 2026-02-22T20:54:22.541 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46750.log 2026-02-22T20:54:22.541 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64523.log.gz 2026-02-22T20:54:22.541 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79800.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79800.log.gz 2026-02-22T20:54:22.541 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44655.log 2026-02-22T20:54:22.541 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46750.log.gz 2026-02-22T20:54:22.541 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65445.log 2026-02-22T20:54:22.541 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44655.log.gz 2026-02-22T20:54:22.541 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92135.log 2026-02-22T20:54:22.542 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65445.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65445.log.gz 2026-02-22T20:54:22.542 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89677.log 2026-02-22T20:54:22.542 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92135.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92135.log.gz 2026-02-22T20:54:22.542 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88602.log 2026-02-22T20:54:22.542 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89677.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89677.log.gz 2026-02-22T20:54:22.542 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49362.log 2026-02-22T20:54:22.542 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88602.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88602.log.gz 2026-02-22T20:54:22.542 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52380.log 2026-02-22T20:54:22.543 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49362.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49362.log.gz 2026-02-22T20:54:22.543 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95511.log 2026-02-22T20:54:22.543 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52380.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52380.log.gz 2026-02-22T20:54:22.543 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105003.log 2026-02-22T20:54:22.543 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95511.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95511.log.gz 2026-02-22T20:54:22.543 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83587.log 2026-02-22T20:54:22.544 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105003.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105003.log.gz 2026-02-22T20:54:22.544 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51714.log 2026-02-22T20:54:22.544 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83587.log.gz 2026-02-22T20:54:22.544 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64984.log 2026-02-22T20:54:22.544 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51714.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51714.log.gz 2026-02-22T20:54:22.544 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70464.log 2026-02-22T20:54:22.544 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64984.log.gz 2026-02-22T20:54:22.544 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99535.log 2026-02-22T20:54:22.545 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70464.log.gz 2026-02-22T20:54:22.545 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94488.log 2026-02-22T20:54:22.545 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99535.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99535.log.gz 2026-02-22T20:54:22.545 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50436.log 2026-02-22T20:54:22.545 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94488.log.gz 2026-02-22T20:54:22.545 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106949.log 2026-02-22T20:54:22.545 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50436.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50436.log.gz 2026-02-22T20:54:22.545 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94796.log 2026-02-22T20:54:22.549 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106949.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106949.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83742.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94796.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90598.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83742.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87579.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90598.log.gzgzip 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.109974.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87579.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87579.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75738.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109974.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87066.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75738.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75738.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49153.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87066.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87066.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60071.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49153.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61351.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60071.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60071.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58076.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61351.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42385.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80823.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58076.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42385.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92595.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80823.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109508.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64831.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92595.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109508.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109508.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61094.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64676.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64831.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61094.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86964.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94181.log 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64676.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86964.log.gz 2026-02-22T20:54:22.550 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54183.log 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49923.log 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94181.log.gz 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54183.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54183.log.gz 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107051.log 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77805.log 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49923.log.gz 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107051.log.gz 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58024.log 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79032.log 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77805.log.gz 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58024.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58024.log.gz 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89320.log 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65343.log 2026-02-22T20:54:22.551 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79032.log.gz 2026-02-22T20:54:22.552 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89320.log.gz 2026-02-22T20:54:22.552 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71337.log 2026-02-22T20:54:22.552 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82770.log 2026-02-22T20:54:22.552 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65343.log.gz 2026-02-22T20:54:22.552 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71337.log.gz 2026-02-22T20:54:22.552 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52738.log 2026-02-22T20:54:22.552 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82770.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82770.log.gz 2026-02-22T20:54:22.552 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109098.log 2026-02-22T20:54:22.552 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52738.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52738.log.gz 2026-02-22T20:54:22.552 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70975.log 2026-02-22T20:54:22.553 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86094.log 2026-02-22T20:54:22.553 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109098.log.gz 2026-02-22T20:54:22.553 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70975.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70975.log.gz 2026-02-22T20:54:22.553 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106640.log 2026-02-22T20:54:22.553 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86094.log.gz 2026-02-22T20:54:22.553 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60173.log 2026-02-22T20:54:22.553 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106640.log.gz 2026-02-22T20:54:22.553 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80160.log 2026-02-22T20:54:22.554 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60173.log.gz 2026-02-22T20:54:22.554 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85275.log 2026-02-22T20:54:22.554 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80160.log.gz 2026-02-22T20:54:22.554 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89473.log 2026-02-22T20:54:22.554 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85275.log.gz 2026-02-22T20:54:22.554 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77237.log 2026-02-22T20:54:22.554 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89473.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89473.log.gz 2026-02-22T20:54:22.554 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80979.log 2026-02-22T20:54:22.554 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77237.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77237.log.gz 2026-02-22T20:54:22.555 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75534.log 2026-02-22T20:54:22.555 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80979.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80979.log.gz 2026-02-22T20:54:22.555 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103570.log 2026-02-22T20:54:22.555 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75534.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75534.log.gz 2026-02-22T20:54:22.555 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66471.log 2026-02-22T20:54:22.555 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103570.log.gz 2026-02-22T20:54:22.555 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-22T20:54:22.555 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65087.log 2026-02-22T20:54:22.555 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66471.log.gz 2026-02-22T20:54:22.556 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97302.log 2026-02-22T20:54:22.556 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65087.log.gz 2026-02-22T20:54:22.556 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89984.log 2026-02-22T20:54:22.556 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97302.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.68519.log 2026-02-22T20:54:22.556 INFO:teuthology.orchestra.run.trial084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97302.log.gz 2026-02-22T20:54:22.556 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89984.log.gz 2026-02-22T20:54:22.557 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74613.log 2026-02-22T20:54:22.557 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61913.log 2026-02-22T20:54:22.557 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68519.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68519.log.gz 2026-02-22T20:54:22.557 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74613.log.gz 2026-02-22T20:54:22.557 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64267.log 2026-02-22T20:54:22.557 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5/var/log/ceph/ceph-client.admin.61913.log: --verbose -- /var/log/ceph/ceph-client.admin.84405.log 2026-02-22T20:54:22.557 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61913.log.gz 2026-02-22T20:54:22.558 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64267.log.gz 2026-02-22T20:54:22.558 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69441.log 2026-02-22T20:54:22.558 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.84405.log: /var/log/ceph/ceph-client.admin.59866.log 2026-02-22T20:54:22.558 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84405.log.gz 2026-02-22T20:54:22.558 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69441.log.gz 2026-02-22T20:54:22.558 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82923.log 2026-02-22T20:54:22.558 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59866.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59866.log.gz -5 2026-02-22T20:54:22.558 INFO:teuthology.orchestra.run.trial084.stderr: --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-22T20:54:22.559 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106129.log 2026-02-22T20:54:22.559 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82923.log.gz 2026-02-22T20:54:22.559 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73231.log 2026-02-22T20:54:22.559 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53761.log 2026-02-22T20:54:22.559 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106129.log.gz 2026-02-22T20:54:22.559 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52482.log 2026-02-22T20:54:22.560 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73231.log.gz 2026-02-22T20:54:22.560 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64574.log 2026-02-22T20:54:22.560 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53761.log.gz 2026-02-22T20:54:22.560 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101028.log 2026-02-22T20:54:22.560 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52482.log.gz 2026-02-22T20:54:22.560 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64574.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64574.log.gz 2026-02-22T20:54:22.560 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105770.log 2026-02-22T20:54:22.561 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.101028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62527.log 2026-02-22T20:54:22.561 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101028.log.gz 2026-02-22T20:54:22.561 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94335.log 2026-02-22T20:54:22.561 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105770.log.gz 2026-02-22T20:54:22.561 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69594.log 2026-02-22T20:54:22.561 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62527.log.gz 2026-02-22T20:54:22.561 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50844.log 2026-02-22T20:54:22.562 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94335.log.gz 2026-02-22T20:54:22.562 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99747.log 2026-02-22T20:54:22.562 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69594.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65394.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50844.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48130.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99747.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65547.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65394.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56947.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48130.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66062.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65547.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80364.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56947.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96279.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66062.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77186.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80364.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95870.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96279.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77084.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77186.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92186.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95870.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74562.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77084.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103212.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92186.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100955.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74562.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45930.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103212.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77856.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100955.log.gz 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78009.log 2026-02-22T20:54:22.565 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45930.log.gz 2026-02-22T20:54:22.566 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87220.log 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77856.log.gz 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84663.log 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78009.log.gz 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53812.log 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87220.log.gz 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91878.log 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84663.log.gz 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78471.log 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53812.log.gz 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107102.log 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91878.log.gz 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67085.log 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78471.log.gz 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53199.log 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107102.log.gz 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53403.log 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67085.log.gz 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69798.log 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53199.log.gz 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50793.log 2026-02-22T20:54:22.567 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53403.log.gz 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99213.log 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69798.log.gz 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75074.log 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50793.log.gz 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78522.log 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99213.log.gz 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51153.log 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75074.log.gz 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58486.log 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78522.log.gz 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47056.log 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51153.log.gz 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58486.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31505.log 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58486.log.gz 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53046.log 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47056.log.gz 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph.tmp-client.admin.31505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71439.log 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31505.log.gz 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93773.log 2026-02-22T20:54:22.568 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53046.log.gz 2026-02-22T20:54:22.569 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80874.log 2026-02-22T20:54:22.569 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71439.log.gz 2026-02-22T20:54:22.569 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107562.log 2026-02-22T20:54:22.569 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93773.log.gz 2026-02-22T20:54:22.569 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100260.log 2026-02-22T20:54:22.569 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80874.log.gz 2026-02-22T20:54:22.569 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55720.log 2026-02-22T20:54:22.569 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107562.log.gz 2026-02-22T20:54:22.569 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57868.log 2026-02-22T20:54:22.569 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100260.log.gz 2026-02-22T20:54:22.570 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55720.log.gz 2026-02-22T20:54:22.570 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56692.log 2026-02-22T20:54:22.570 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96485.log 2026-02-22T20:54:22.570 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57868.log.gz 2026-02-22T20:54:22.570 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77033.log 2026-02-22T20:54:22.570 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56692.log.gz 2026-02-22T20:54:22.570 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68621.log 2026-02-22T20:54:22.570 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96485.log.gz 2026-02-22T20:54:22.571 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66420.log 2026-02-22T20:54:22.571 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77033.log.gz 2026-02-22T20:54:22.571 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88960.log 2026-02-22T20:54:22.571 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68621.log.gz 2026-02-22T20:54:22.571 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71592.log 2026-02-22T20:54:22.571 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66420.log.gz 2026-02-22T20:54:22.571 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51561.log 2026-02-22T20:54:22.571 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88960.log.gz 2026-02-22T20:54:22.571 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105463.log 2026-02-22T20:54:22.571 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71592.log.gz 2026-02-22T20:54:22.571 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99029.log 2026-02-22T20:54:22.571 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51561.log.gz 2026-02-22T20:54:22.572 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74050.log 2026-02-22T20:54:22.572 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105463.log.gz 2026-02-22T20:54:22.572 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58639.log 2026-02-22T20:54:22.572 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99029.log.gz 2026-02-22T20:54:22.572 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66573.log 2026-02-22T20:54:22.572 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74050.log.gz 2026-02-22T20:54:22.572 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66369.log 2026-02-22T20:54:22.572 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58639.log.gz 2026-02-22T20:54:22.572 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81387.log 2026-02-22T20:54:22.572 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66573.log.gz 2026-02-22T20:54:22.572 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71797.log 2026-02-22T20:54:22.572 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66369.log.gz 2026-02-22T20:54:22.573 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81387.log.gz 2026-02-22T20:54:22.573 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61504.log 2026-02-22T20:54:22.573 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60583.log 2026-02-22T20:54:22.573 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71797.log.gz 2026-02-22T20:54:22.573 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88857.log 2026-02-22T20:54:22.573 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61504.log.gz 2026-02-22T20:54:22.573 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51816.log 2026-02-22T20:54:22.573 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60583.log.gz 2026-02-22T20:54:22.574 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88857.log.gz 2026-02-22T20:54:22.574 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107665.log 2026-02-22T20:54:22.574 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51816.log.gz 2026-02-22T20:54:22.574 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93363.log 2026-02-22T20:54:22.574 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50334.log 2026-02-22T20:54:22.574 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107665.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107665.log.gz 2026-02-22T20:54:22.574 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84969.log 2026-02-22T20:54:22.574 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93363.log.gz 2026-02-22T20:54:22.574 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110281.log 2026-02-22T20:54:22.574 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50334.log.gz 2026-02-22T20:54:22.574 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54542.log 2026-02-22T20:54:22.575 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84969.log.gz 2026-02-22T20:54:22.575 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76263.log 2026-02-22T20:54:22.575 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110281.log.gz 2026-02-22T20:54:22.575 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54542.log.gz 2026-02-22T20:54:22.575 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105106.log 2026-02-22T20:54:22.575 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72769.log 2026-02-22T20:54:22.575 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76263.log.gz 2026-02-22T20:54:22.575 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77550.log 2026-02-22T20:54:22.575 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105106.log.gz 2026-02-22T20:54:22.575 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109252.log 2026-02-22T20:54:22.575 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72769.log.gz 2026-02-22T20:54:22.576 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94745.log 2026-02-22T20:54:22.576 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77550.log.gz 2026-02-22T20:54:22.576 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66932.log 2026-02-22T20:54:22.576 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109252.log.gz 2026-02-22T20:54:22.576 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94745.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-22T20:54:22.576 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94745.log.gz 2026-02-22T20:54:22.576 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105668.log 2026-02-22T20:54:22.576 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66932.log.gz 2026-02-22T20:54:22.576 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59356.log 2026-02-22T20:54:22.576 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106487.log 2026-02-22T20:54:22.577 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105668.log.gz 2026-02-22T20:54:22.577 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64472.log 2026-02-22T20:54:22.577 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59356.log.gz 2026-02-22T20:54:22.577 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88449.log 2026-02-22T20:54:22.577 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106487.log.gz 2026-02-22T20:54:22.577 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64472.log.gz 2026-02-22T20:54:22.577 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67955.log 2026-02-22T20:54:22.578 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95564.log 2026-02-22T20:54:22.578 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88449.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88449.log.gz 2026-02-22T20:54:22.578 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96995.log 2026-02-22T20:54:22.578 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67955.log.gz 2026-02-22T20:54:22.578 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84612.log 2026-02-22T20:54:22.578 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95564.log.gz 2026-02-22T20:54:22.579 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57152.log 2026-02-22T20:54:22.579 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96995.log.gz 2026-02-22T20:54:22.579 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106078.log 2026-02-22T20:54:22.579 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84612.log.gz 2026-02-22T20:54:22.579 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81030.log 2026-02-22T20:54:22.579 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57152.log.gz 2026-02-22T20:54:22.579 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45471.log 2026-02-22T20:54:22.579 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106078.log.gz 2026-02-22T20:54:22.579 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73742.log 2026-02-22T20:54:22.579 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81030.log.gz 2026-02-22T20:54:22.579 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76161.log 2026-02-22T20:54:22.579 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45471.log.gz 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49872.log 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73742.log.gz 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87426.log 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76161.log.gz 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103315.log 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49872.log.gz 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85429.log 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87426.log.gz 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50076.log 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103315.log.gz 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75483.log 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85429.log.gz 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78624.log 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50076.log.gz 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56998.log 2026-02-22T20:54:22.580 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75483.log.gz 2026-02-22T20:54:22.581 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67188.log 2026-02-22T20:54:22.581 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78624.log.gz 2026-02-22T20:54:22.581 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70362.log 2026-02-22T20:54:22.581 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56998.log.gz 2026-02-22T20:54:22.581 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50691.log 2026-02-22T20:54:22.581 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67188.log.gz 2026-02-22T20:54:22.581 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109303.log 2026-02-22T20:54:22.581 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70362.log.gz 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88653.log 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50691.log.gz 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60020.log 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109303.log.gz 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85020.log 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88653.log.gz 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101079.log 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60020.log.gz 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106538.log 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85020.log.gz 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.101079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43700.log 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101079.log.gz 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69645.log 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106538.log.gz 2026-02-22T20:54:22.582 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83484.log 2026-02-22T20:54:22.583 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43700.log.gz 2026-02-22T20:54:22.583 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46134.log 2026-02-22T20:54:22.583 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69645.log.gz 2026-02-22T20:54:22.583 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73026.log 2026-02-22T20:54:22.583 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83484.log.gz 2026-02-22T20:54:22.583 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100414.log 2026-02-22T20:54:22.583 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46134.log.gz 2026-02-22T20:54:22.583 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75636.log 2026-02-22T20:54:22.583 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73026.log.gz 2026-02-22T20:54:22.584 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104133.log 2026-02-22T20:54:22.584 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100414.log.gz 2026-02-22T20:54:22.584 INFO:teuthology.orchestra.run.trial084.stderr:gzip/var/log/ceph/ceph-client.admin.75636.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.79443.log 2026-02-22T20:54:22.584 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75636.log.gz 2026-02-22T20:54:22.584 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66113.log 2026-02-22T20:54:22.584 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104133.log.gz 2026-02-22T20:54:22.584 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85686.log 2026-02-22T20:54:22.584 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79443.log.gz 2026-02-22T20:54:22.584 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42777.log 2026-02-22T20:54:22.584 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66113.log.gz 2026-02-22T20:54:22.584 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47875.log 2026-02-22T20:54:22.584 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85686.log.gz 2026-02-22T20:54:22.585 INFO:teuthology.orchestra.run.trial084.stderr:gzip/var/log/ceph/ceph-client.admin.42777.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48592.log 2026-02-22T20:54:22.585 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42777.log.gz 2026-02-22T20:54:22.585 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100186.log 2026-02-22T20:54:22.585 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47875.log.gz 2026-02-22T20:54:22.585 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69030.log 2026-02-22T20:54:22.585 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48592.log.gz 2026-02-22T20:54:22.585 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67904.log 2026-02-22T20:54:22.585 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100186.log.gz 2026-02-22T20:54:22.585 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57100.log 2026-02-22T20:54:22.585 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69030.log.gz 2026-02-22T20:54:22.585 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81591.log 2026-02-22T20:54:22.585 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67904.log.gz 2026-02-22T20:54:22.586 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52944.log 2026-02-22T20:54:22.586 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57100.log.gz 2026-02-22T20:54:22.586 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56794.log 2026-02-22T20:54:22.586 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81591.log.gz 2026-02-22T20:54:22.586 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38877.log 2026-02-22T20:54:22.586 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52944.log.gz 2026-02-22T20:54:22.586 INFO:teuthology.orchestra.run.trial084.stderr:gzip/var/log/ceph/ceph-client.admin.56794.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45267.log 2026-02-22T20:54:22.586 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56794.log.gz 2026-02-22T20:54:22.586 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.38877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63193.log 2026-02-22T20:54:22.586 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38877.log.gz 2026-02-22T20:54:22.586 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50998.log 2026-02-22T20:54:22.587 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45267.log.gz 2026-02-22T20:54:22.587 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81081.log 2026-02-22T20:54:22.587 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63193.log.gz 2026-02-22T20:54:22.587 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45420.log 2026-02-22T20:54:22.587 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50998.log.gz 2026-02-22T20:54:22.587 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80262.log 2026-02-22T20:54:22.587 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81081.log.gz 2026-02-22T20:54:22.587 INFO:teuthology.orchestra.run.trial084.stderr:gzip/var/log/ceph/ceph-client.admin.45420.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.67649.log 2026-02-22T20:54:22.587 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45420.log.gz 2026-02-22T20:54:22.587 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68364.log 2026-02-22T20:54:22.587 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80262.log.gz 2026-02-22T20:54:22.588 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73077.log 2026-02-22T20:54:22.588 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67649.log.gz 2026-02-22T20:54:22.588 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45726.log 2026-02-22T20:54:22.588 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68364.log.gz 2026-02-22T20:54:22.588 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107613.log 2026-02-22T20:54:22.588 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73077.log.gz 2026-02-22T20:54:22.588 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71541.log 2026-02-22T20:54:22.588 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45726.log.gz 2026-02-22T20:54:22.588 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64727.log 2026-02-22T20:54:22.588 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107613.log.gz 2026-02-22T20:54:22.588 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74921.log 2026-02-22T20:54:22.589 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71541.log.gz 2026-02-22T20:54:22.589 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99799.log 2026-02-22T20:54:22.589 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64727.log.gz 2026-02-22T20:54:22.589 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71128.log 2026-02-22T20:54:22.589 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74921.log.gz 2026-02-22T20:54:22.589 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108483.log 2026-02-22T20:54:22.589 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99799.log.gz 2026-02-22T20:54:22.589 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43437.log 2026-02-22T20:54:22.589 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71128.log.gz 2026-02-22T20:54:22.590 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68876.log 2026-02-22T20:54:22.590 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108483.log.gz 2026-02-22T20:54:22.590 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70003.log 2026-02-22T20:54:22.590 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43437.log.gz 2026-02-22T20:54:22.590 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67341.log 2026-02-22T20:54:22.590 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68876.log.gz 2026-02-22T20:54:22.590 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43290.log 2026-02-22T20:54:22.590 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70003.log.gz 2026-02-22T20:54:22.590 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73948.log 2026-02-22T20:54:22.590 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67341.log.gz 2026-02-22T20:54:22.590 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65906.log 2026-02-22T20:54:22.590 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43290.log.gz 2026-02-22T20:54:22.591 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96689.log 2026-02-22T20:54:22.591 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73948.log.gz 2026-02-22T20:54:22.591 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93516.log 2026-02-22T20:54:22.591 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65906.log.gz 2026-02-22T20:54:22.591 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91982.log 2026-02-22T20:54:22.591 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96689.log.gz 2026-02-22T20:54:22.591 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71950.log 2026-02-22T20:54:22.591 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93516.log.gz 2026-02-22T20:54:22.591 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53877.log 2026-02-22T20:54:22.591 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91982.log.gz 2026-02-22T20:54:22.591 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74307.log 2026-02-22T20:54:22.592 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71950.log.gz 2026-02-22T20:54:22.592 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98235.log 2026-02-22T20:54:22.592 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53877.log.gz 2026-02-22T20:54:22.592 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92852.log 2026-02-22T20:54:22.592 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74307.log.gz 2026-02-22T20:54:22.592 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53301.log 2026-02-22T20:54:22.592 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98235.log.gz 2026-02-22T20:54:22.592 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61453.log 2026-02-22T20:54:22.592 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92852.log.gz 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85482.log 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53301.log.gz 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48181.log 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61453.log.gz 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85482.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85482.log.gz 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72616.log 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100465.log 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48181.log.gz 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100084.log 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72616.log.gz 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95615.log 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100465.log.gz 2026-02-22T20:54:22.593 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58894.log 2026-02-22T20:54:22.594 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100084.log.gz 2026-02-22T20:54:22.594 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108586.log 2026-02-22T20:54:22.594 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95615.log.gz 2026-02-22T20:54:22.594 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84303.log 2026-02-22T20:54:22.594 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58894.log.gz 2026-02-22T20:54:22.594 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80211.log 2026-02-22T20:54:22.594 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108586.log.gz 2026-02-22T20:54:22.594 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60890.log 2026-02-22T20:54:22.594 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84303.log.gz 2026-02-22T20:54:22.595 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98031.log 2026-02-22T20:54:22.595 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80211.log.gz 2026-02-22T20:54:22.595 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83178.log 2026-02-22T20:54:22.595 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60890.log.gz 2026-02-22T20:54:22.595 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51663.log 2026-02-22T20:54:22.595 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98031.log.gz 2026-02-22T20:54:22.595 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51612.log 2026-02-22T20:54:22.595 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83178.log.gz 2026-02-22T20:54:22.595 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93720.log 2026-02-22T20:54:22.595 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51663.log.gz 2026-02-22T20:54:22.595 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74204.log 2026-02-22T20:54:22.595 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51612.log.gz 2026-02-22T20:54:22.596 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77703.log 2026-02-22T20:54:22.596 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93720.log.gz 2026-02-22T20:54:22.596 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5/var/log/ceph/ceph-client.admin.74204.log: --verbose -- /var/log/ceph/ceph-client.admin.82665.log 2026-02-22T20:54:22.596 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74204.log.gz 2026-02-22T20:54:22.596 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54593.log 2026-02-22T20:54:22.596 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77703.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77703.log.gz 2026-02-22T20:54:22.596 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103468.log 2026-02-22T20:54:22.596 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82665.log.gz 2026-02-22T20:54:22.596 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88551.log 2026-02-22T20:54:22.596 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54593.log.gz 2026-02-22T20:54:22.597 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103468.log.gz 2026-02-22T20:54:22.597 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86709.log 2026-02-22T20:54:22.597 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46648.log 2026-02-22T20:54:22.597 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88551.log.gz 2026-02-22T20:54:22.597 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94694.log 2026-02-22T20:54:22.597 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86709.log.gz 2026-02-22T20:54:22.597 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100055.log 2026-02-22T20:54:22.597 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46648.log.gz 2026-02-22T20:54:22.597 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97251.log/var/log/ceph/ceph-client.admin.94694.log: 2026-02-22T20:54:22.597 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94694.log.gz 2026-02-22T20:54:22.598 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48847.log 2026-02-22T20:54:22.598 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100055.log.gz 2026-02-22T20:54:22.598 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45879.log 2026-02-22T20:54:22.598 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97251.log.gz 2026-02-22T20:54:22.598 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50025.log 2026-02-22T20:54:22.598 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48847.log.gz 2026-02-22T20:54:22.598 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78318.log 2026-02-22T20:54:22.598 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45879.log.gz 2026-02-22T20:54:22.598 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50025.log: -- /var/log/ceph/ceph-client.admin.77754.log 2026-02-22T20:54:22.598 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50025.log.gz 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78369.log 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78318.log.gz 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72207.log 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77754.log.gz 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101130.log 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78369.log.gz 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73180.log 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72207.log.gz 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.101130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95307.log 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101130.log.gz 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71899.log/var/log/ceph/ceph-client.admin.73180.log: 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73180.log.gz 2026-02-22T20:54:22.599 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90292.log 2026-02-22T20:54:22.600 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95307.log.gz 2026-02-22T20:54:22.600 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55156.log 2026-02-22T20:54:22.600 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71899.log.gz 2026-02-22T20:54:22.600 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106027.log 2026-02-22T20:54:22.600 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90292.log.gz 2026-02-22T20:54:22.600 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60634.log 2026-02-22T20:54:22.600 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55156.log.gz 2026-02-22T20:54:22.600 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54490.log 2026-02-22T20:54:22.600 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106027.log.gz 2026-02-22T20:54:22.600 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77339.log 2026-02-22T20:54:22.601 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60634.log.gz 2026-02-22T20:54:22.601 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59048.log 2026-02-22T20:54:22.601 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54490.log.gz 2026-02-22T20:54:22.601 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.77339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78111.log 2026-02-22T20:54:22.601 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.77339.log.gz 2026-02-22T20:54:22.601 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51357.log 2026-02-22T20:54:22.601 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59048.log.gz 2026-02-22T20:54:22.601 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85173.log 2026-02-22T20:54:22.601 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78111.log.gz 2026-02-22T20:54:22.601 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57817.log 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51357.log.gz 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42999.log 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85173.log.gz 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60685.log 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57817.log.gz 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106691.log 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42999.log.gz 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86403.log 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60685.log.gz 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76982.log 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106691.log.gz 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84048.log 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86403.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86403.log.gz 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83025.log 2026-02-22T20:54:22.602 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76982.log.gz 2026-02-22T20:54:22.603 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67137.log 2026-02-22T20:54:22.603 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84048.log.gz 2026-02-22T20:54:22.603 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109923.log 2026-02-22T20:54:22.603 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83025.log.gz 2026-02-22T20:54:22.603 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67137.log.gz 2026-02-22T20:54:22.603 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92237.log 2026-02-22T20:54:22.603 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98775.log 2026-02-22T20:54:22.603 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109923.log.gz 2026-02-22T20:54:22.604 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68109.log 2026-02-22T20:54:22.604 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92237.log.gz 2026-02-22T20:54:22.604 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54695.log 2026-02-22T20:54:22.604 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98775.log.gz 2026-02-22T20:54:22.604 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67290.log 2026-02-22T20:54:22.604 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68109.log.gz 2026-02-22T20:54:22.604 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104542.log 2026-02-22T20:54:22.604 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54695.log.gz 2026-02-22T20:54:22.604 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86556.log 2026-02-22T20:54:22.604 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67290.log.gz 2026-02-22T20:54:22.604 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74153.log 2026-02-22T20:54:22.604 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104542.log.gz 2026-02-22T20:54:22.605 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86556.log.gz 2026-02-22T20:54:22.605 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95921.log 2026-02-22T20:54:22.605 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74153.log.gz 2026-02-22T20:54:22.605 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65804.log 2026-02-22T20:54:22.605 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95921.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95921.log.gz 2026-02-22T20:54:22.605 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44910.log 2026-02-22T20:54:22.605 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65804.log.gz 2026-02-22T20:54:22.605 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47416.log 2026-02-22T20:54:22.605 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44910.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44910.log.gz 2026-02-22T20:54:22.605 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99937.log 2026-02-22T20:54:22.606 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47416.log.gz 2026-02-22T20:54:22.606 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79698.log 2026-02-22T20:54:22.606 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99937.log.gz 2026-02-22T20:54:22.606 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44706.log 2026-02-22T20:54:22.606 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79698.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79698.log.gz 2026-02-22T20:54:22.606 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87936.log 2026-02-22T20:54:22.606 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44706.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44706.log.gz 2026-02-22T20:54:22.606 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93567.log 2026-02-22T20:54:22.607 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87936.log.gz 2026-02-22T20:54:22.607 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89882.log 2026-02-22T20:54:22.607 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93567.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93567.log.gz 2026-02-22T20:54:22.607 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49413.log 2026-02-22T20:54:22.607 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89882.log.gz 2026-02-22T20:54:22.607 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73691.log 2026-02-22T20:54:22.607 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49413.log.gz 2026-02-22T20:54:22.607 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76622.log 2026-02-22T20:54:22.608 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87271.log 2026-02-22T20:54:22.608 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73691.log.gz 2026-02-22T20:54:22.608 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76622.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76622.log.gz 2026-02-22T20:54:22.608 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63755.log 2026-02-22T20:54:22.608 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87271.log.gz 2026-02-22T20:54:22.608 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85533.log 2026-02-22T20:54:22.608 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63755.log.gz 2026-02-22T20:54:22.608 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74511.log 2026-02-22T20:54:22.609 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85533.log.gz 2026-02-22T20:54:22.609 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75585.log 2026-02-22T20:54:22.609 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74511.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74511.log.gz 2026-02-22T20:54:22.609 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2026-02-22T20:54:22.609 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75585.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75585.log.gz 2026-02-22T20:54:22.609 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96536.log 2026-02-22T20:54:22.609 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108228.log 2026-02-22T20:54:22.610 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96536.log.gz 2026-02-22T20:54:22.610 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79545.log 2026-02-22T20:54:22.610 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108228.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108228.log.gz 2026-02-22T20:54:22.610 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61555.log 2026-02-22T20:54:22.610 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79545.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79545.log.gz 2026-02-22T20:54:22.610 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106180.log 2026-02-22T20:54:22.610 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61555.log.gz 2026-02-22T20:54:22.611 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52125.log 2026-02-22T20:54:22.611 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106180.log.gz 2026-02-22T20:54:22.611 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56486.log 2026-02-22T20:54:22.611 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52125.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52125.log.gz 2026-02-22T20:54:22.611 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38079.log 2026-02-22T20:54:22.611 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56486.log.gz 2026-02-22T20:54:22.611 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99936.log 2026-02-22T20:54:22.611 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.38079.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38079.log.gz 2026-02-22T20:54:22.612 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70260.log 2026-02-22T20:54:22.612 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99936.log.gz 2026-02-22T20:54:22.612 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70924.log 2026-02-22T20:54:22.612 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70260.log.gz 2026-02-22T20:54:22.612 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42334.log 2026-02-22T20:54:22.612 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70924.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70924.log.gz 2026-02-22T20:54:22.612 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110486.log 2026-02-22T20:54:22.612 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42334.log.gz 2026-02-22T20:54:22.613 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98937.log 2026-02-22T20:54:22.613 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110486.log.gz 2026-02-22T20:54:22.613 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56230.log 2026-02-22T20:54:22.613 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98937.log.gz 2026-02-22T20:54:22.613 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107511.log 2026-02-22T20:54:22.613 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56230.log.gz 2026-02-22T20:54:22.613 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87117.log 2026-02-22T20:54:22.614 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83793.log 2026-02-22T20:54:22.614 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107511.log.gz 2026-02-22T20:54:22.614 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87117.log.gz 2026-02-22T20:54:22.614 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92084.log 2026-02-22T20:54:22.614 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83793.log.gz 2026-02-22T20:54:22.614 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70311.log 2026-02-22T20:54:22.614 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92084.log.gz 2026-02-22T20:54:22.614 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78060.log 2026-02-22T20:54:22.615 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70311.log.gz 2026-02-22T20:54:22.615 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44429.log 2026-02-22T20:54:22.615 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78060.log.gz 2026-02-22T20:54:22.615 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71286.log 2026-02-22T20:54:22.615 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44429.log.gz 2026-02-22T20:54:22.615 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44036.log 2026-02-22T20:54:22.615 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71286.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71286.log.gz 2026-02-22T20:54:22.615 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90547.log 2026-02-22T20:54:22.616 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44036.log.gz 2026-02-22T20:54:22.616 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52789.log 2026-02-22T20:54:22.616 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109457.log 2026-02-22T20:54:22.616 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90547.log.gz 2026-02-22T20:54:22.616 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52789.log.gz 2026-02-22T20:54:22.616 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-22T20:54:22.617 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93669.log 2026-02-22T20:54:22.617 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109457.log.gz 2026-02-22T20:54:22.617 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48949.log 2026-02-22T20:54:22.617 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90956.log 2026-02-22T20:54:22.617 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93669.log.gz 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97661.log 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48949.log.gz 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69747.log 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90956.log.gz 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86350.log 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97661.log.gz 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96074.log 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69747.log.gz 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97814.log 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86350.log.gz 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94847.log 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96074.log.gz 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68927.log 2026-02-22T20:54:22.618 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97814.log.gz 2026-02-22T20:54:22.619 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94847.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94847.log.gz 2026-02-22T20:54:22.619 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42283.log 2026-02-22T20:54:22.619 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96023.log 2026-02-22T20:54:22.619 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68927.log.gz 2026-02-22T20:54:22.619 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84508.log 2026-02-22T20:54:22.619 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42283.log.gz 2026-02-22T20:54:22.619 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75791.log 2026-02-22T20:54:22.619 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96023.log.gz 2026-02-22T20:54:22.619 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70515.log 2026-02-22T20:54:22.619 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84508.log.gz 2026-02-22T20:54:22.620 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45777.log 2026-02-22T20:54:22.620 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75791.log.gz 2026-02-22T20:54:22.620 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96740.log 2026-02-22T20:54:22.620 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70515.log.gz 2026-02-22T20:54:22.620 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48079.log 2026-02-22T20:54:22.620 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45777.log.gz 2026-02-22T20:54:22.620 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110537.log 2026-02-22T20:54:22.620 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96740.log.gz 2026-02-22T20:54:22.620 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57203.log 2026-02-22T20:54:22.620 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48079.log.gz 2026-02-22T20:54:22.621 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48439.log 0.0% 2026-02-22T20:54:22.621 INFO:teuthology.orchestra.run.trial084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.110537.log.gz 2026-02-22T20:54:22.621 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81744.log 2026-02-22T20:54:22.621 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57203.log.gz 2026-02-22T20:54:22.621 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50895.log 2026-02-22T20:54:22.621 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48439.log.gz 2026-02-22T20:54:22.621 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67034.log 2026-02-22T20:54:22.621 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81744.log.gz 2026-02-22T20:54:22.621 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71181.log 2026-02-22T20:54:22.621 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50895.log.gz 2026-02-22T20:54:22.622 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63551.log 2026-02-22T20:54:22.622 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67034.log.gz 2026-02-22T20:54:22.622 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98393.log 2026-02-22T20:54:22.622 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71181.log.gz 2026-02-22T20:54:22.622 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100750.log 2026-02-22T20:54:22.622 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63551.log.gz 2026-02-22T20:54:22.622 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37815.log 2026-02-22T20:54:22.622 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98393.log.gz 2026-02-22T20:54:22.622 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58588.log 2026-02-22T20:54:22.622 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100750.log.gz 2026-02-22T20:54:22.622 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.37815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85224.log 2026-02-22T20:54:22.623 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37815.log.gz 2026-02-22T20:54:22.623 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66830.log 2026-02-22T20:54:22.623 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58588.log.gz 2026-02-22T20:54:22.623 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88141.log 2026-02-22T20:54:22.623 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85224.log.gz 2026-02-22T20:54:22.623 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103775.log 2026-02-22T20:54:22.623 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66830.log.gz 2026-02-22T20:54:22.623 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88500.log 2026-02-22T20:54:22.623 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88141.log.gz 2026-02-22T20:54:22.623 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59611.log 2026-02-22T20:54:22.623 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103775.log.gz 2026-02-22T20:54:22.624 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59969.log 2026-02-22T20:54:22.624 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88500.log.gz 2026-02-22T20:54:22.624 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80721.log 2026-02-22T20:54:22.624 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59611.log.gz 2026-02-22T20:54:22.624 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69696.log 2026-02-22T20:54:22.624 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59969.log.gz 2026-02-22T20:54:22.624 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88090.log 2026-02-22T20:54:22.624 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80721.log.gz 2026-02-22T20:54:22.624 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96638.log 2026-02-22T20:54:22.624 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69696.log.gz 2026-02-22T20:54:22.624 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47773.log 2026-02-22T20:54:22.624 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88090.log.gz 2026-02-22T20:54:22.625 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87834.log 2026-02-22T20:54:22.625 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96638.log.gz 2026-02-22T20:54:22.625 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109354.log 2026-02-22T20:54:22.625 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47773.log.gz 2026-02-22T20:54:22.625 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94079.log 2026-02-22T20:54:22.625 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87834.log.gz 2026-02-22T20:54:22.625 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.109354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79596.log 2026-02-22T20:54:22.625 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109354.log.gz 2026-02-22T20:54:22.625 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44181.log 2026-02-22T20:54:22.626 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94079.log.gz 2026-02-22T20:54:22.626 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54644.log 2026-02-22T20:54:22.626 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79596.log.gz 2026-02-22T20:54:22.626 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98495.log 2026-02-22T20:54:22.626 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44181.log.gz 2026-02-22T20:54:22.626 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65649.log 2026-02-22T20:54:22.626 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54644.log.gz 2026-02-22T20:54:22.626 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42948.log 2026-02-22T20:54:22.626 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98495.log.gz 2026-02-22T20:54:22.626 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64625.log 2026-02-22T20:54:22.626 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65649.log.gz 2026-02-22T20:54:22.626 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93875.log 2026-02-22T20:54:22.627 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42948.log.gz 2026-02-22T20:54:22.628 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60787.log 2026-02-22T20:54:22.628 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64625.log.gz 2026-02-22T20:54:22.628 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65496.log 2026-02-22T20:54:22.628 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93875.log.gz 2026-02-22T20:54:22.628 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60379.log 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60787.log.gz 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61657.log 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65496.log.gz 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70156.log 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60379.log.gz 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82308.log 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61657.log.gz 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.70156.log: /var/log/ceph/ceph-client.admin.110128.log 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70156.log.gz 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38236.log 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82308.log.gz 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56282.log 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110128.log.gz 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.38236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44859.log 2026-02-22T20:54:22.629 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38236.log.gz 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91315.log 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56282.log.gz 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46390.log 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44859.log.gz 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55616.log 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91315.log.gz 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59150.log 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46390.log.gz 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80313.log 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55616.log.gz 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76777.log 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59150.log.gz 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63244.log 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80313.log.gz 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94949.log 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76777.log.gz 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96228.log 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63244.log.gz 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108125.log 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94949.log.gz 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43438.log 2026-02-22T20:54:22.640 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96228.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88296.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108125.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98548.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43438.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67802.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88296.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52329.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98548.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96944.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67802.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88245.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52329.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93618.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96944.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99673.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88245.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.93618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54901.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93618.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52431.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99673.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55975.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54901.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87987.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52431.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52431.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68160.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55975.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49974.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87987.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106333.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68160.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60328.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49974.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53659.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106333.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45624.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60328.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52891.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53659.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83433.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45624.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57715.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52891.log.gz 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87324.log 2026-02-22T20:54:22.641 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83433.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56179.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57715.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57766.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87324.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88908.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56179.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104900.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57766.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92697.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88908.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86043.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104900.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63091.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92697.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5/var/log/ceph/ceph-client.admin.86043.log: --verbose -- /var/log/ceph/ceph-client.admin.89831.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86043.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64165.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63091.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72001.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89831.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42573.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64165.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76569.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72001.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86247.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42573.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68416.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76569.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94590.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86247.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80928.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68416.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.94590.log: -- /var/log/ceph/ceph-client.admin.97559.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94590.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43291.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80928.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72258.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97559.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88347.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43291.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110640.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72258.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78828.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88347.log.gz 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79239.log 2026-02-22T20:54:22.642 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110640.log.gz 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66318.log 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78828.log.gz 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44808.log 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79239.log.gz 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45675.log 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66318.log.gz 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45522.log 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44808.log.gz 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89166.log 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45675.log.gz 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50742.log 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45522.log.gz 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55771.log 2026-02-22T20:54:22.643 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89166.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50742.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81183.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110230.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55771.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55771.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56896.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81183.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110230.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108534.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84765.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56896.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56896.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104952.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108534.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98340.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84765.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84765.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51459.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104952.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104952.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107255.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98340.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51459.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52584.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48232.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107255.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96893.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52584.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104236.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48232.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44327.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96893.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99935.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104236.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76007.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44327.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82359.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99935.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57562.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76007.log.gz 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104645.log 2026-02-22T20:54:22.646 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82359.log.gz 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73538.log 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57562.log.gz 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92544.log 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104645.log.gz 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98287.log 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73538.log.gz 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107819.log 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92544.log.gz 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106436.log 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98287.log.gz 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84150.log 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107819.log.gz 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87528.log 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106436.log.gz 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106794.log 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84150.log.gz 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46903.log 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87528.log.gz 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80415.log 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106794.log.gz 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52635.log 2026-02-22T20:54:22.647 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46903.log.gz 2026-02-22T20:54:22.648 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42214.log 2026-02-22T20:54:22.648 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80415.log.gz 2026-02-22T20:54:22.648 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95256.log 2026-02-22T20:54:22.648 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52635.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98701.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42214.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84867.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95256.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57664.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98701.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84867.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84867.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98184.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59713.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57664.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57664.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85122.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98184.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44961.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59713.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53097.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85122.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45012.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44961.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83844.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53097.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95102.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45012.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97712.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83844.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83844.log.gz 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48541.log 2026-02-22T20:54:22.649 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95102.log.gz 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81336.log 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97712.log.gz 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76212.log 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48541.log.gz 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65753.log 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81336.log.gz 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83946.log 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76212.log.gz 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48337.log 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65753.log.gz 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80006.log 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83946.log.gz 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75687.log 2026-02-22T20:54:22.650 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48337.log.gz 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53454.log 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80006.log.gz 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91725.log 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75687.log.gz 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96791.log 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53454.log.gz 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75944.log 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91725.log.gz 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70719.log 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96791.log.gz 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89371.log 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75944.log.gz 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76467.log 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70719.log.gz 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47005.log 2026-02-22T20:54:22.651 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89371.log.gz 2026-02-22T20:54:22.652 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106743.log 2026-02-22T20:54:22.652 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76467.log.gz 2026-02-22T20:54:22.652 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67598.log 2026-02-22T20:54:22.652 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47005.log.gz 2026-02-22T20:54:22.652 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58331.log 2026-02-22T20:54:22.652 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106743.log.gz 2026-02-22T20:54:22.652 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78981.log 2026-02-22T20:54:22.652 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67598.log.gz 2026-02-22T20:54:22.652 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72309.log 2026-02-22T20:54:22.652 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58331.log.gz 2026-02-22T20:54:22.652 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47209.log 2026-02-22T20:54:22.652 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78981.log.gz 2026-02-22T20:54:22.653 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63961.log 2026-02-22T20:54:22.653 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72309.log.gz 2026-02-22T20:54:22.653 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107716.log 2026-02-22T20:54:22.653 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47209.log.gz 2026-02-22T20:54:22.653 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57509.log 2026-02-22T20:54:22.653 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63961.log.gz 2026-02-22T20:54:22.653 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64933.log 2026-02-22T20:54:22.653 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107716.log.gz 2026-02-22T20:54:22.654 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90701.log 2026-02-22T20:54:22.654 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57509.log.gz 2026-02-22T20:54:22.654 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64933.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64933.log.gz 2026-02-22T20:54:22.654 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89011.log 2026-02-22T20:54:22.654 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49617.log 2026-02-22T20:54:22.654 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90701.log.gz 2026-02-22T20:54:22.654 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75023.log 2026-02-22T20:54:22.654 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89011.log.gz 2026-02-22T20:54:22.654 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49617.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.49617.log.gz -- 2026-02-22T20:54:22.654 INFO:teuthology.orchestra.run.trial084.stderr: /var/log/ceph/ceph-client.admin.55360.log 2026-02-22T20:54:22.654 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78420.log 2026-02-22T20:54:22.654 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75023.log.gz 2026-02-22T20:54:22.655 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78215.log 2026-02-22T20:54:22.655 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55360.log.gz 2026-02-22T20:54:22.655 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83076.log 2026-02-22T20:54:22.655 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78420.log.gz 2026-02-22T20:54:22.655 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61862.log 2026-02-22T20:54:22.655 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78215.log.gz 2026-02-22T20:54:22.655 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104849.log 2026-02-22T20:54:22.655 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83076.log.gz 2026-02-22T20:54:22.655 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64318.log 2026-02-22T20:54:22.656 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61862.log.gz 2026-02-22T20:54:22.656 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44502.log 2026-02-22T20:54:22.656 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104849.log.gz 2026-02-22T20:54:22.656 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107767.log 2026-02-22T20:54:22.656 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64318.log.gz 2026-02-22T20:54:22.656 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74766.log 2026-02-22T20:54:22.656 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44502.log.gz 2026-02-22T20:54:22.656 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75330.log 2026-02-22T20:54:22.656 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107767.log.gz 2026-02-22T20:54:22.656 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63346.log 2026-02-22T20:54:22.656 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74766.log.gz 2026-02-22T20:54:22.657 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51919.log 2026-02-22T20:54:22.657 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75330.log.gz 2026-02-22T20:54:22.657 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42779.log 2026-02-22T20:54:22.657 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63346.log.gz 2026-02-22T20:54:22.657 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51255.log 2026-02-22T20:54:22.657 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51919.log.gz 2026-02-22T20:54:22.657 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82461.log 2026-02-22T20:54:22.657 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42779.log.gz 2026-02-22T20:54:22.657 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95717.log 2026-02-22T20:54:22.657 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51255.log.gz 2026-02-22T20:54:22.657 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72411.log 2026-02-22T20:54:22.657 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82461.log.gz 2026-02-22T20:54:22.658 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47620.log 2026-02-22T20:54:22.658 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95717.log.gz 2026-02-22T20:54:22.658 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104696.log 2026-02-22T20:54:22.658 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72411.log.gz 2026-02-22T20:54:22.658 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96177.log 2026-02-22T20:54:22.658 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47620.log.gz 2026-02-22T20:54:22.658 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46801.log 2026-02-22T20:54:22.658 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104696.log.gz 2026-02-22T20:54:22.658 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66009.log 2026-02-22T20:54:22.658 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96177.log.gz 2026-02-22T20:54:22.659 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44178.log 2026-02-22T20:54:22.659 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46801.log.gz 2026-02-22T20:54:22.659 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66009.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-22T20:54:22.659 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66009.log.gz 2026-02-22T20:54:22.659 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.44178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87783.log 2026-02-22T20:54:22.659 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44178.log.gz 2026-02-22T20:54:22.659 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52176.log 2026-02-22T20:54:22.659 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97865.log 2026-02-22T20:54:22.659 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87783.log.gz 2026-02-22T20:54:22.660 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52176.log.gz 2026-02-22T20:54:22.660 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55463.log 2026-02-22T20:54:22.660 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.97865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97865.log.gz 2026-02-22T20:54:22.660 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56641.log 2026-02-22T20:54:22.660 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89779.log 2026-02-22T20:54:22.660 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55463.log.gz 2026-02-22T20:54:22.660 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57613.log 2026-02-22T20:54:22.660 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56641.log.gz 2026-02-22T20:54:22.661 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61402.log 2026-02-22T20:54:22.661 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89779.log.gz 2026-02-22T20:54:22.661 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52227.log 2026-02-22T20:54:22.661 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57613.log.gz 2026-02-22T20:54:22.661 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45165.log 2026-02-22T20:54:22.661 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61402.log.gz 2026-02-22T20:54:22.661 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.52227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81438.log 2026-02-22T20:54:22.661 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52227.log.gz 2026-02-22T20:54:22.661 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63704.log 2026-02-22T20:54:22.661 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45165.log.gz 2026-02-22T20:54:22.661 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51970.log 2026-02-22T20:54:22.661 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81438.log.gz 2026-02-22T20:54:22.662 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85788.log 2026-02-22T20:54:22.662 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63704.log.gz 2026-02-22T20:54:22.662 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98082.log 2026-02-22T20:54:22.662 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51970.log.gz 2026-02-22T20:54:22.662 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90803.log 2026-02-22T20:54:22.662 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85788.log.gz 2026-02-22T20:54:22.662 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69543.log 2026-02-22T20:54:22.662 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98082.log.gz 2026-02-22T20:54:22.662 INFO:teuthology.orchestra.run.trial084.stderr: 96.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-22T20:54:22.662 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74664.log 2026-02-22T20:54:22.662 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90803.log.gz 2026-02-22T20:54:22.663 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49309.log 2026-02-22T20:54:22.663 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69543.log.gz 2026-02-22T20:54:22.663 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95460.log 2026-02-22T20:54:22.663 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74664.log.gz 2026-02-22T20:54:22.663 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67239.log 2026-02-22T20:54:22.663 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49309.log.gz 2026-02-22T20:54:22.663 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61300.log 2026-02-22T20:54:22.663 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95460.log.gz 2026-02-22T20:54:22.663 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74409.log 2026-02-22T20:54:22.663 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67239.log.gz 2026-02-22T20:54:22.663 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61300.log.gz 2026-02-22T20:54:22.663 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61198.log 2026-02-22T20:54:22.664 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95768.log 2026-02-22T20:54:22.664 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74409.log.gz 2026-02-22T20:54:22.664 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61198.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-22T20:54:22.664 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61198.log.gz 2026-02-22T20:54:22.664 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42775.log 2026-02-22T20:54:22.664 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95768.log.gz 2026-02-22T20:54:22.664 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53505.log 2026-02-22T20:54:22.664 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94437.log 2026-02-22T20:54:22.664 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42775.log.gz 2026-02-22T20:54:22.664 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73793.log 2026-02-22T20:54:22.665 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53505.log.gz 2026-02-22T20:54:22.665 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88192.log 2026-02-22T20:54:22.665 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94437.log.gz 2026-02-22T20:54:22.665 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92339.log 2026-02-22T20:54:22.665 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73793.log.gz 2026-02-22T20:54:22.665 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65189.log 2026-02-22T20:54:22.665 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88192.log.gz 2026-02-22T20:54:22.665 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65701.log 2026-02-22T20:54:22.665 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92339.log.gz 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105873.log 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65189.log.gz 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.65701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91417.log 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65701.log.gz 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54285.log 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105873.log.gz 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108843.log 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91417.log.gz 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104491.log 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54285.log.gz 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr: 96.0% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64063.log 2026-02-22T20:54:22.666 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108843.log.gz 2026-02-22T20:54:22.667 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91009.log 2026-02-22T20:54:22.667 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104491.log.gz 2026-02-22T20:54:22.667 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.64063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66267.log 2026-02-22T20:54:22.667 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64063.log.gz 2026-02-22T20:54:22.667 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80670.log 2026-02-22T20:54:22.667 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91009.log.gz 2026-02-22T20:54:22.667 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54030.log 2026-02-22T20:54:22.667 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66267.log.gz 2026-02-22T20:54:22.667 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.80670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50128.log 2026-02-22T20:54:22.668 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80670.log.gz 2026-02-22T20:54:22.668 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85635.log 2026-02-22T20:54:22.668 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54030.log.gz 2026-02-22T20:54:22.668 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73589.log 2026-02-22T20:54:22.668 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50128.log.gz 2026-02-22T20:54:22.668 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72718.log 2026-02-22T20:54:22.668 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85635.log.gz 2026-02-22T20:54:22.668 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95000.log 2026-02-22T20:54:22.668 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73589.log.gz 2026-02-22T20:54:22.668 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108945.log 2026-02-22T20:54:22.668 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72718.log.gz 2026-02-22T20:54:22.669 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62119.log 2026-02-22T20:54:22.669 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95000.log.gz 2026-02-22T20:54:22.669 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.108945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54336.log 2026-02-22T20:54:22.669 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108945.log.gz 2026-02-22T20:54:22.669 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42896.log 2026-02-22T20:54:22.669 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62119.log.gz 2026-02-22T20:54:22.669 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81234.log 2026-02-22T20:54:22.669 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54336.log.gz 2026-02-22T20:54:22.669 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.42896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85737.log 2026-02-22T20:54:22.669 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42896.log.gz 2026-02-22T20:54:22.669 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95819.log 2026-02-22T20:54:22.669 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81234.log.gz 2026-02-22T20:54:22.670 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68006.log 2026-02-22T20:54:22.670 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85737.log.gz 2026-02-22T20:54:22.670 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53352.log 2026-02-22T20:54:22.670 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95819.log.gz 2026-02-22T20:54:22.670 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106845.log 2026-02-22T20:54:22.670 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68006.log.gz 2026-02-22T20:54:22.670 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38359.log 2026-02-22T20:54:22.670 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53352.log.gz 2026-02-22T20:54:22.670 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76726.log 2026-02-22T20:54:22.670 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106845.log.gz 2026-02-22T20:54:22.671 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.38359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86196.log 2026-02-22T20:54:22.671 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38359.log.gz 2026-02-22T20:54:22.671 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92903.log 2026-02-22T20:54:22.671 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76726.log.gz 2026-02-22T20:54:22.671 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58997.log 2026-02-22T20:54:22.671 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86196.log.gz 2026-02-22T20:54:22.671 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56026.log 2026-02-22T20:54:22.671 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92903.log.gz 2026-02-22T20:54:22.672 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50640.log 2026-02-22T20:54:22.672 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58997.log.gz 2026-02-22T20:54:22.672 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.56026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58127.log 2026-02-22T20:54:22.672 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56026.log.gz 2026-02-22T20:54:22.672 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.50640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100157.log 2026-02-22T20:54:22.672 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50640.log.gz 2026-02-22T20:54:22.672 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.58127.log: -- /var/log/ceph/ceph-client.admin.100853.log 2026-02-22T20:54:22.672 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58127.log.gz 2026-02-22T20:54:22.672 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46546.log 2026-02-22T20:54:22.672 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100157.log.gz 2026-02-22T20:54:22.672 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100853.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100853.log.gz 2026-02-22T20:54:22.672 INFO:teuthology.orchestra.run.trial084.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99259.log 2026-02-22T20:54:22.673 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92441.log 2026-02-22T20:54:22.673 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46546.log.gz 2026-02-22T20:54:22.673 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68570.log 2026-02-22T20:54:22.673 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99259.log.gz 2026-02-22T20:54:22.673 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.92441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105412.log 2026-02-22T20:54:22.673 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92441.log.gz 2026-02-22T20:54:22.673 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54952.log 2026-02-22T20:54:22.673 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68570.log.gz 2026-02-22T20:54:22.673 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73487.log 2026-02-22T20:54:22.673 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105412.log.gz 2026-02-22T20:54:22.673 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82155.log 2026-02-22T20:54:22.674 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54952.log.gz 2026-02-22T20:54:22.674 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94232.log 2026-02-22T20:54:22.674 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73487.log.gz 2026-02-22T20:54:22.674 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83895.log 2026-02-22T20:54:22.674 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82155.log.gz 2026-02-22T20:54:22.674 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.94232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95051.log 2026-02-22T20:54:22.674 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94232.log.gz 2026-02-22T20:54:22.674 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59099.log 2026-02-22T20:54:22.674 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83895.log.gz 2026-02-22T20:54:22.675 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.95051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68468.log 2026-02-22T20:54:22.675 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95051.log.gz 2026-02-22T20:54:22.675 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48490.log 2026-02-22T20:54:22.675 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59099.log.gz 2026-02-22T20:54:22.675 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82206.log 2026-02-22T20:54:22.675 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68468.log.gz 2026-02-22T20:54:22.675 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106282.log 2026-02-22T20:54:22.675 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48490.log.gz 2026-02-22T20:54:22.675 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70413.log 2026-02-22T20:54:22.675 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82206.log.gz 2026-02-22T20:54:22.675 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.106282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85377.log 2026-02-22T20:54:22.675 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106282.log.gz 2026-02-22T20:54:22.676 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99443.log 2026-02-22T20:54:22.676 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70413.log.gz 2026-02-22T20:54:22.676 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53928.log 2026-02-22T20:54:22.676 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85377.log.gz 2026-02-22T20:54:22.676 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.99443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89217.log 2026-02-22T20:54:22.676 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99443.log.gz 2026-02-22T20:54:22.676 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67547.log 2026-02-22T20:54:22.676 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53928.log.gz 2026-02-22T20:54:22.676 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.89217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71644.log 2026-02-22T20:54:22.676 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89217.log.gz 2026-02-22T20:54:22.676 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48283.log 2026-02-22T20:54:22.676 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67547.log.gz 2026-02-22T20:54:22.677 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49668.log 2026-02-22T20:54:22.677 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71644.log.gz 2026-02-22T20:54:22.677 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.48283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53607.log 2026-02-22T20:54:22.677 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48283.log.gz 2026-02-22T20:54:22.677 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79392.log 2026-02-22T20:54:22.677 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49668.log.gz 2026-02-22T20:54:22.677 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104287.log 2026-02-22T20:54:22.677 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53607.log.gz 2026-02-22T20:54:22.677 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62476.log 2026-02-22T20:54:22.677 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79392.log.gz 2026-02-22T20:54:22.678 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78930.log 2026-02-22T20:54:22.678 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104287.log.gz 2026-02-22T20:54:22.678 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82053.log 2026-02-22T20:54:22.678 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62476.log.gz 2026-02-22T20:54:22.678 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.78930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76059.log 2026-02-22T20:54:22.678 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.78930.log.gz 2026-02-22T20:54:22.678 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.82053.log: /var/log/ceph/ceph-client.admin.81642.log 2026-02-22T20:54:22.678 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82053.log.gz 2026-02-22T20:54:22.678 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46492.log 2026-02-22T20:54:22.678 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76059.log.gz 2026-02-22T20:54:22.678 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79085.log 2026-02-22T20:54:22.678 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81642.log.gz 2026-02-22T20:54:22.679 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57407.log/var/log/ceph/ceph-client.admin.46492.log: 2026-02-22T20:54:22.679 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46492.log.gz 2026-02-22T20:54:22.679 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68825.log 2026-02-22T20:54:22.679 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79085.log.gz 2026-02-22T20:54:22.679 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63448.log 2026-02-22T20:54:22.679 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57407.log.gz 2026-02-22T20:54:22.679 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.68825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91623.log 2026-02-22T20:54:22.679 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68825.log.gz 2026-02-22T20:54:22.679 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70668.log 2026-02-22T20:54:22.679 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63448.log.gz 2026-02-22T20:54:22.679 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60839.log 2026-02-22T20:54:22.679 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91623.log.gz 2026-02-22T20:54:22.680 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98983.log 2026-02-22T20:54:22.680 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70668.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70668.log.gz 2026-02-22T20:54:22.680 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43496.log 2026-02-22T20:54:22.680 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60839.log.gz 2026-02-22T20:54:22.680 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.98983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43287.log 2026-02-22T20:54:22.680 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98983.log.gz 2026-02-22T20:54:22.680 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76879.log 2026-02-22T20:54:22.680 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43496.log.gz 2026-02-22T20:54:22.680 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86505.log 2026-02-22T20:54:22.680 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43287.log.gz 2026-02-22T20:54:22.680 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60941.log 2026-02-22T20:54:22.680 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76879.log.gz 2026-02-22T20:54:22.681 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76416.log 2026-02-22T20:54:22.681 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86505.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86505.log.gz 2026-02-22T20:54:22.681 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58741.log 2026-02-22T20:54:22.681 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60941.log.gz 2026-02-22T20:54:22.681 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.76416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107204.log 2026-02-22T20:54:22.681 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.76416.log.gz 2026-02-22T20:54:22.681 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53979.log 2026-02-22T20:54:22.681 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58741.log.gz 2026-02-22T20:54:22.682 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55105.log 2026-02-22T20:54:22.682 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107204.log.gz 2026-02-22T20:54:22.682 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.53979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100363.log 2026-02-22T20:54:22.682 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53979.log.gz 2026-02-22T20:54:22.682 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84201.log 2026-02-22T20:54:22.682 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55105.log.gz 2026-02-22T20:54:22.682 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.100363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49000.log 2026-02-22T20:54:22.682 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100363.log.gz 2026-02-22T20:54:22.682 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.84201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63806.log 2026-02-22T20:54:22.682 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84201.log.gz 2026-02-22T20:54:22.682 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61146.log 2026-02-22T20:54:22.682 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49000.log.gz 2026-02-22T20:54:22.683 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57973.log 2026-02-22T20:54:22.683 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63806.log.gz 2026-02-22T20:54:22.683 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54234.log 2026-02-22T20:54:22.683 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61146.log.gz 2026-02-22T20:54:22.683 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.57973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105054.log 2026-02-22T20:54:22.683 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57973.log.gz 2026-02-22T20:54:22.683 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72871.log 2026-02-22T20:54:22.683 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54234.log.gz 2026-02-22T20:54:22.683 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66522.log 2026-02-22T20:54:22.683 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105054.log.gz 2026-02-22T20:54:22.684 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104389.log 2026-02-22T20:54:22.684 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72871.log.gz 2026-02-22T20:54:22.684 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63500.log 2026-02-22T20:54:22.684 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66522.log.gz 2026-02-22T20:54:22.684 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.104389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45318.log 2026-02-22T20:54:22.684 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104389.log.gz 2026-02-22T20:54:22.684 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38487.log 2026-02-22T20:54:22.684 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63500.log.gz 2026-02-22T20:54:22.684 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96381.log 2026-02-22T20:54:22.684 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.45318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45318.log.gz 2026-02-22T20:54:22.684 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.38487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62067.log 2026-02-22T20:54:22.684 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38487.log.gz 2026-02-22T20:54:22.685 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73895.log 2026-02-22T20:54:22.685 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96381.log.gz 2026-02-22T20:54:22.685 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51204.log 2026-02-22T20:54:22.685 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62067.log.gz 2026-02-22T20:54:22.685 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.73895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46954.log 2026-02-22T20:54:22.685 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73895.log.gz 2026-02-22T20:54:22.685 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.51204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60736.log 2026-02-22T20:54:22.685 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51204.log.gz 2026-02-22T20:54:22.685 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49566.log/var/log/ceph/ceph-client.admin.46954.log: 2026-02-22T20:54:22.685 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46954.log.gz 2026-02-22T20:54:22.685 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59662.log 2026-02-22T20:54:22.685 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60736.log.gz 2026-02-22T20:54:22.686 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.49566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74870.log 2026-02-22T20:54:22.686 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49566.log.gz 2026-02-22T20:54:22.686 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.59662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46185.log 2026-02-22T20:54:22.686 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59662.log.gz 2026-02-22T20:54:22.686 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.74870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79188.log 2026-02-22T20:54:22.686 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74870.log.gz 2026-02-22T20:54:22.686 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.46185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86607.log 2026-02-22T20:54:22.686 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46185.log.gz 2026-02-22T20:54:22.686 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75125.log 2026-02-22T20:54:22.686 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79188.log.gz 2026-02-22T20:54:22.686 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.86607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86607.log.gz 2026-02-22T20:54:22.687 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103417.log 2026-02-22T20:54:22.687 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94028.log 2026-02-22T20:54:22.687 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.75125.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.75125.log.gz 2026-02-22T20:54:22.687 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79647.log 2026-02-22T20:54:22.687 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103417.log.gz 2026-02-22T20:54:22.687 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5/var/log/ceph/ceph-client.admin.94028.log: --verbose -- /var/log/ceph/ceph-client.admin.70207.log 2026-02-22T20:54:22.687 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94028.log.gz 2026-02-22T20:54:22.687 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.79647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63295.log 2026-02-22T20:54:22.687 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.79647.log.gz 2026-02-22T20:54:22.687 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69132.log 2026-02-22T20:54:22.688 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70207.log.gz 2026-02-22T20:54:22.688 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67751.log 2026-02-22T20:54:22.688 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63295.log.gz 2026-02-22T20:54:22.688 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110332.log 2026-02-22T20:54:22.688 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69132.log.gz 2026-02-22T20:54:22.688 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.67751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71077.log 2026-02-22T20:54:22.688 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67751.log.gz 2026-02-22T20:54:22.688 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66881.log 2026-02-22T20:54:22.688 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110332.log.gz 2026-02-22T20:54:22.688 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.71077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60430.log 2026-02-22T20:54:22.688 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71077.log.gz 2026-02-22T20:54:22.688 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.66881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103673.log 2026-02-22T20:54:22.689 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66881.log.gz 2026-02-22T20:54:22.689 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.60430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107358.log 2026-02-22T20:54:22.689 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60430.log.gz 2026-02-22T20:54:22.689 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.103673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47362.log 2026-02-22T20:54:22.689 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103673.log.gz 2026-02-22T20:54:22.689 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105719.log 2026-02-22T20:54:22.689 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107358.log.gz 2026-02-22T20:54:22.689 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96587.log 2026-02-22T20:54:22.689 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47362.log.gz 2026-02-22T20:54:22.690 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82512.log 2026-02-22T20:54:22.690 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105719.log.gz 2026-02-22T20:54:22.690 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91213.log 2026-02-22T20:54:22.690 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96587.log.gz 2026-02-22T20:54:22.690 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72667.log 2026-02-22T20:54:22.690 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82512.log.gz 2026-02-22T20:54:22.690 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.91213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105566.log 2026-02-22T20:54:22.690 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91213.log.gz 2026-02-22T20:54:22.690 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.72667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62579.log 2026-02-22T20:54:22.690 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72667.log.gz 2026-02-22T20:54:22.690 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.105566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85326.log 2026-02-22T20:54:22.690 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105566.log.gz 2026-02-22T20:54:22.691 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.62579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110077.log 2026-02-22T20:54:22.691 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62579.log.gz 2026-02-22T20:54:22.691 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.85326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43641.log 2026-02-22T20:54:22.691 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85326.log.gz 2026-02-22T20:54:22.691 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.110077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83229.log 2026-02-22T20:54:22.691 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110077.log.gz 2026-02-22T20:54:22.691 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.43641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107921.log 2026-02-22T20:54:22.691 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43641.log.gz 2026-02-22T20:54:22.691 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.83229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63858.log 2026-02-22T20:54:22.691 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83229.log.gz 2026-02-22T20:54:22.691 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.107921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69183.log 2026-02-22T20:54:22.691 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107921.log.gz 2026-02-22T20:54:22.692 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70617.log 2026-02-22T20:54:22.692 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63858.log.gz 2026-02-22T20:54:22.692 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.69183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88755.log 2026-02-22T20:54:22.692 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69183.log.gz 2026-02-22T20:54:22.692 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.70617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90445.log 2026-02-22T20:54:22.692 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70617.log.gz 2026-02-22T20:54:22.692 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.88755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61709.log 2026-02-22T20:54:22.692 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88755.log.gz 2026-02-22T20:54:22.692 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.90445.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-22T20:54:22.692 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90445.log.gz 2026-02-22T20:54:22.692 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82002.log 2026-02-22T20:54:22.692 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61709.log.gz 2026-02-22T20:54:22.693 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58229.log 2026-02-22T20:54:22.693 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.82002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56845.log 2026-02-22T20:54:22.693 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82002.log.gz 2026-02-22T20:54:22.693 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.58229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54081.log 2026-02-22T20:54:22.693 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58229.log.gz 2026-02-22T20:54:22.693 INFO:teuthology.orchestra.run.trial084.stderr:gzip/var/log/ceph/ceph-client.admin.56845.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47518.log 2026-02-22T20:54:22.693 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56845.log.gz 2026-02-22T20:54:22.693 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.54081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81951.log 2026-02-22T20:54:22.693 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54081.log.gz 2026-02-22T20:54:22.694 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.47518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55309.log 2026-02-22T20:54:22.694 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47518.log.gz 2026-02-22T20:54:22.694 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.81951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87732.log 2026-02-22T20:54:22.694 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.81951.log.gz 2026-02-22T20:54:22.694 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.55309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70054.log 2026-02-22T20:54:22.694 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55309.log.gz 2026-02-22T20:54:22.694 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.87732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63909.log 2026-02-22T20:54:22.694 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87732.log.gz 2026-02-22T20:54:22.694 INFO:teuthology.orchestra.run.trial084.stderr:gzip/var/log/ceph/ceph-client.admin.70054.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.61606.log 2026-02-22T20:54:22.694 INFO:teuthology.orchestra.run.trial084.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70054.log.gz 2026-02-22T20:54:22.694 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96433.log 2026-02-22T20:54:22.695 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.63909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63909.log.gz 2026-02-22T20:54:22.695 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.61606.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61606.log.gz 2026-02-22T20:54:22.695 INFO:teuthology.orchestra.run.trial084.stderr:/var/log/ceph/ceph-client.admin.96433.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96433.log.gz 2026-02-22T20:54:22.708 INFO:teuthology.orchestra.run.trial084.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2026-02-22T20:54:22.797 INFO:teuthology.orchestra.run.trial084.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2026-02-22T20:54:22.905 INFO:teuthology.orchestra.run.trial084.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-22T20:54:40.336 INFO:teuthology.orchestra.run.trial084.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-22T20:54:48.237 INFO:teuthology.orchestra.run.trial084.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-22T20:54:50.105 INFO:teuthology.orchestra.run.trial084.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-22T20:56:13.926 INFO:teuthology.orchestra.run.trial084.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-22T20:56:13.928 INFO:teuthology.orchestra.run.trial084.stderr: 2026-02-22T20:56:13.928 INFO:teuthology.orchestra.run.trial084.stderr:real 1m51.537s 2026-02-22T20:56:13.928 INFO:teuthology.orchestra.run.trial084.stderr:user 2m54.552s 2026-02-22T20:56:13.928 INFO:teuthology.orchestra.run.trial084.stderr:sys 0m8.685s 2026-02-22T20:56:13.929 INFO:tasks.ceph:Archiving logs... 2026-02-22T20:56:13.931 DEBUG:teuthology.misc:Transferring archived files from trial084:/var/log/ceph to /home/teuthworker/mnt/teuthology/nmordech-2026-02-22_19:44:01-rados-wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle-distro-default-trial/64522/remote/trial084/log 2026-02-22T20:56:13.932 DEBUG:teuthology.orchestra.run.trial084:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-22T20:56:34.206 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-22T20:56:34.222 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-22T20:56:34.222 DEBUG:teuthology.orchestra.run.trial084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-22T20:56:34.331 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-22T20:56:34.342 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-22T20:56:34.342 DEBUG:teuthology.orchestra.run.trial084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T20:56:34.360 INFO:teuthology.orchestra.run.trial084.stderr:bash: line 1: ntpq: command not found 2026-02-22T20:56:34.382 INFO:teuthology.orchestra.run.trial084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-22T20:56:34.382 INFO:teuthology.orchestra.run.trial084.stdout:=============================================================================== 2026-02-22T20:56:34.382 INFO:teuthology.orchestra.run.trial084.stdout:^* ntp.nyc.icanbwell.com 2 6 377 58 -164us[ -497us] +/- 2865us 2026-02-22T20:56:34.382 INFO:teuthology.orchestra.run.trial084.stdout:^- 144.202.41.38.vultruserc> 2 7 377 123 +1555us[ +936us] +/- 36ms 2026-02-22T20:56:34.382 INFO:teuthology.orchestra.run.trial084.stdout:^- ntp11.kernfusion.at 2 7 377 59 -286us[ -619us] +/- 72ms 2026-02-22T20:56:34.382 INFO:teuthology.orchestra.run.trial084.stdout:^- ntp.maxhost.io 2 7 377 124 +326us[ -293us] +/- 56ms 2026-02-22T20:56:34.383 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-22T20:56:34.395 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-22T20:56:34.395 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-22T20:56:34.409 DEBUG:teuthology.orchestra.run.trial084:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-22T20:56:34.492 DEBUG:teuthology.orchestra.run.trial084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e 'comm="systemd".*denied.*\{ prog_run \}.*tclass=bpf.*permissive=1' 2026-02-22T20:56:34.512 INFO:teuthology.orchestra.run.trial084.stderr:grep: Invalid content of \{\} 2026-02-22T20:56:34.524 DEBUG:teuthology.orchestra.run:got remote process result: 2 2026-02-22T20:56:34.524 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-22T20:56:34.543 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-22T20:56:34.565 INFO:teuthology.task.internal:Duration was 1650.796701 seconds 2026-02-22T20:56:34.566 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-22T20:56:34.585 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-22T20:56:34.586 DEBUG:teuthology.orchestra.run.trial084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-22T20:56:34.626 INFO:teuthology.orchestra.run.trial084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-22T20:56:34.748 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-22T20:56:34.748 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial084.front.sepia.ceph.com 2026-02-22T20:56:34.748 DEBUG:teuthology.orchestra.run.trial084:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-22T20:56:34.809 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-22T20:56:34.810 DEBUG:teuthology.orchestra.run.trial084:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T20:56:34.925 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-22T20:56:34.925 DEBUG:teuthology.orchestra.run.trial084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T20:56:34.952 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T20:56:34.952 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T20:56:34.952 INFO:teuthology.orchestra.run.trial084.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-22T20:56:34.953 INFO:teuthology.orchestra.run.trial084.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T20:56:34.953 INFO:teuthology.orchestra.run.trial084.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-22T20:56:34.963 INFO:teuthology.orchestra.run.trial084.stderr: 87.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-22T20:56:34.965 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-22T20:56:34.982 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-22T20:56:34.982 DEBUG:teuthology.orchestra.run.trial084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-22T20:56:35.031 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-22T20:56:35.044 DEBUG:teuthology.orchestra.run.trial084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-22T20:56:35.116 INFO:teuthology.orchestra.run.trial084.stdout:kernel.core_pattern = core 2026-02-22T20:56:35.130 DEBUG:teuthology.orchestra.run.trial084:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-22T20:56:35.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T20:56:35.145 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-22T20:56:35.157 INFO:teuthology.task.internal:Transferring archived files... 2026-02-22T20:56:35.157 DEBUG:teuthology.misc:Transferring archived files from trial084:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/nmordech-2026-02-22_19:44:01-rados-wip-rocky10-branch-of-the-day-2026-02-22-1771776513-tentacle-distro-default-trial/64522/remote/trial084 2026-02-22T20:56:35.157 DEBUG:teuthology.orchestra.run.trial084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-22T20:56:35.222 INFO:teuthology.task.internal:Removing archive directory... 2026-02-22T20:56:35.222 DEBUG:teuthology.orchestra.run.trial084:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-22T20:56:35.274 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-22T20:56:35.292 INFO:teuthology.task.internal:Not uploading archives. 2026-02-22T20:56:35.292 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-22T20:56:35.307 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-22T20:56:35.307 DEBUG:teuthology.orchestra.run.trial084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-22T20:56:35.332 INFO:teuthology.orchestra.run.trial084.stdout: 786901 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 22 20:56 /home/ubuntu/cephtest 2026-02-22T20:56:35.334 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-22T20:56:35.351 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-22T20:56:35.388 INFO:teuthology.run:Summary data: description: rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async objectstore/{bluestore/{alloc$/{hybrid} base mem$/{low} onode-segment$/{1M} write$/{v2/{compr$/{no$/{no}} v2}}}} rados supported-random-distro$/{rpm_latest}} duration: 1650.7967014312744 flavor: default owner: scheduled_nmordech@soko04.front.sepia.ceph.com success: true 2026-02-22T20:56:35.389 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-22T20:56:35.513 INFO:teuthology.run:pass